2024-09-14T22:42:32.345 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T22:42:32.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T22:42:32.462 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} email: ceph-qa@ceph.com first_in_suite: false job_id: '7904734' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: true rbd cache policy: writeback global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4271 sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 sleep_before_teardown: 0 subset: 97/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 targets: smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPekzKbvGGUg4/nEap++Kfd5a3dDcStKT+CG6xu7MaMOVJUPQQxYKVpUwlSOUSspG2pko0Ej8tu4GVzlu6u9Wc= smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEUwbkj0Vp8J0W60rvdzKo/8OLZDKEN00V34UJ2WNb1ovrd07/d981wXeiidBokwOdhTnmv2YLAON9Khoa0lu38= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-14_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T22:42:32.462 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa; will attempt to use it 2024-09-14T22:42:32.462 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa/tasks 2024-09-14T22:42:32.463 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T22:42:32.464 INFO:teuthology.task.internal:Checking packages... 2024-09-14T22:42:32.483 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ecab7be58963a5bd03910cd73eef66405d7fdd50' 2024-09-14T22:42:32.483 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T22:42:32.483 INFO:teuthology.packaging:ref: None 2024-09-14T22:42:32.483 INFO:teuthology.packaging:tag: None 2024-09-14T22:42:32.483 INFO:teuthology.packaging:branch: squid 2024-09-14T22:42:32.483 INFO:teuthology.packaging:sha1: ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:42:32.484 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-14T22:42:32.707 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-324.gecab7be5 2024-09-14T22:42:32.709 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T22:42:32.715 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T22:42:32.715 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T22:42:32.773 INFO:teuthology.task.internal:Saving configuration 2024-09-14T22:42:32.788 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T22:42:32.833 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T22:42:32.859 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 22:35:54.645931', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPekzKbvGGUg4/nEap++Kfd5a3dDcStKT+CG6xu7MaMOVJUPQQxYKVpUwlSOUSspG2pko0Ej8tu4GVzlu6u9Wc='} 2024-09-14T22:42:32.882 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 22:35:54.644396', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEUwbkj0Vp8J0W60rvdzKo/8OLZDKEN00V34UJ2WNb1ovrd07/d981wXeiidBokwOdhTnmv2YLAON9Khoa0lu38='} 2024-09-14T22:42:32.882 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T22:42:32.888 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-14T22:42:32.888 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-14T22:42:32.888 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T22:42:32.964 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe963beb250>, signals=[15]) 2024-09-14T22:42:32.964 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T22:42:32.969 INFO:teuthology.task.internal:Opening connections... 2024-09-14T22:42:32.969 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:42:32.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:42:33.044 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:42:33.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:42:33.121 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T22:42:33.127 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-09-14T22:42:33.144 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-09-14T22:42:33.144 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-09-14T22:42:33.201 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T22:42:33.202 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T22:42:33.203 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T22:42:33.203 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-09-14T22:42:33.225 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-09-14T22:42:33.242 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-09-14T22:42:33.243 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T22:42:33.300 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-09-14T22:42:33.301 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T22:42:33.301 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T22:42:33.301 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T22:42:33.301 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-09-14T22:42:33.324 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T22:42:33.334 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T22:42:33.390 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T22:42:33.390 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-09-14T22:42:33.393 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-09-14T22:42:33.409 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T22:42:33.457 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T22:42:33.458 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-09-14T22:42:33.460 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-09-14T22:42:33.573 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T22:42:33.629 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T22:42:33.645 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T22:42:33.645 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T22:42:33.646 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T22:42:33.646 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T22:42:33.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:42:33.662 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-09-14T22:42:33.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:42:33.663 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-14T22:42:33.720 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-503.el9.x86_64 2024-09-14T22:42:33.720 INFO:teuthology.task.kernel:Running kernel on smithi076: 5.14.0-503.el9.x86_64 2024-09-14T22:42:33.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum install -y kernel 2024-09-14T22:42:33.721 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-503.el9.x86_64 2024-09-14T22:42:33.721 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.14.0-503.el9.x86_64 2024-09-14T22:42:33.721 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum install -y kernel 2024-09-14T22:42:36.058 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-14T22:42:36.157 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-14T22:42:37.926 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-14T22:42:43.568 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-14T22:42:45.027 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-14T22:42:45.981 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-14T22:42:48.507 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 20 MB 00:11 2024-09-14T22:42:51.487 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-14T22:42:53.518 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T22:42:53.519 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T22:42:53.608 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:42:53.616 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:42:53.616 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2024-09-14T22:42:53.616 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:42:53.616 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:42:53.617 INFO:teuthology.orchestra.run.smithi076.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T22:42:53.617 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:42:53.617 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T22:42:53.617 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T22:42:53.617 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T22:42:53.618 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:42:53.618 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:42:53.618 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:42:53.618 INFO:teuthology.orchestra.run.smithi076.stdout:Install 4 Packages 2024-09-14T22:42:53.618 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:42:53.619 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 79 M 2024-09-14T22:42:53.619 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 126 M 2024-09-14T22:42:53.619 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:42:53.988 INFO:teuthology.orchestra.run.smithi076.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 246 kB/s | 23 kB 00:00 2024-09-14T22:42:54.408 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-14T22:42:54.430 INFO:teuthology.orchestra.run.smithi076.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-14T22:42:54.880 INFO:teuthology.orchestra.run.smithi076.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 35 MB/s | 35 MB 00:00 2024-09-14T22:42:55.139 INFO:teuthology.orchestra.run.smithi076.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-14T22:42:55.139 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:42:55.139 INFO:teuthology.orchestra.run.smithi076.stdout:Total 52 MB/s | 79 MB 00:01 2024-09-14T22:42:55.816 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:42:55.917 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-14T22:42:56.178 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:42:56.178 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:42:57.087 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-14T22:42:57.155 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:42:57.156 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:42:59.598 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:43:00.321 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T22:43:00.405 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T22:43:02.410 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-14T22:43:03.211 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T22:43:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T22:43:04.416 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T22:43:04.416 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T22:43:04.502 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:43:04.516 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:43:04.516 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-14T22:43:04.516 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:43:04.516 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-09-14T22:43:04.517 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:43:04.518 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 79 M 2024-09-14T22:43:04.518 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 126 M 2024-09-14T22:43:04.518 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:43:04.644 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 284 kB/s | 23 kB 00:00 2024-09-14T22:43:05.036 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-14T22:43:05.728 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-09-14T22:43:06.070 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-14T22:43:06.070 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:43:06.070 INFO:teuthology.orchestra.run.smithi114.stdout:Total 51 MB/s | 79 MB 00:01 2024-09-14T22:43:06.744 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:43:07.096 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:43:07.097 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:43:08.042 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:43:08.042 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:43:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T22:43:08.271 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:43:10.480 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:43:11.120 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T22:43:11.203 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T22:43:13.146 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:43:13.975 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T22:43:14.105 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T22:43:18.927 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T22:43:19.030 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:43:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:43:51.358 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:43:51.398 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:43:51.778 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:43:51.778 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T22:43:51.778 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T22:43:51.778 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:43:52.662 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:43:53.034 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T22:43:53.121 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T22:43:53.122 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T22:43:53.122 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T22:43:53.122 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T22:43:53.122 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T22:43:53.122 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T22:43:53.122 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T22:43:53.122 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T22:43:53.180 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T22:43:53.180 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T22:43:53.181 DEBUG:teuthology.orchestra.run.smithi076:> sudo rpm -qi grub2-tools 2024-09-14T22:43:53.212 INFO:teuthology.orchestra.run.smithi076.stdout:Name : grub2-tools 2024-09-14T22:43:53.212 INFO:teuthology.orchestra.run.smithi076.stdout:Epoch : 1 2024-09-14T22:43:53.212 INFO:teuthology.orchestra.run.smithi076.stdout:Version : 2.06 2024-09-14T22:43:53.212 INFO:teuthology.orchestra.run.smithi076.stdout:Release : 82.el9 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: x86_64 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Group : Unspecified 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Size : 8274670 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:License : GPLv3+ 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Packager : builder@centos.org 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:Vendor : CentOS 2024-09-14T22:43:53.213 INFO:teuthology.orchestra.run.smithi076.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout:Summary : Support tools for GRUB. 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout:Description : 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout:hardware devices. 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:43:53.214 INFO:teuthology.orchestra.run.smithi076.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T22:43:53.215 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T22:43:53.215 INFO:teuthology.task.kernel:Updating grub on smithi076 to boot 5.14.0-505.el9.x86_64 2024-09-14T22:43:53.215 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T22:43:53.855 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2024-09-14T22:43:55.187 INFO:teuthology.orchestra.run.smithi076.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T22:43:55.224 INFO:teuthology.orchestra.run.smithi076.stderr:done 2024-09-14T22:43:55.226 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-09-14T22:43:55.242 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.4RFUyUndzO 2024-09-14T22:43:55.242 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4RFUyUndzO 2024-09-14T22:43:55.316 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.4RFUyUndzO 2024-09-14T22:43:55.464 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.4RFUyUndzO is 6KB 2024-09-14T22:43:55.485 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.4RFUyUndzO 2024-09-14T22:43:55.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T22:43:55.566 INFO:teuthology.orchestra.run.smithi076.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T22:43:55.566 INFO:teuthology.orchestra.run.smithi076.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T22:43:55.566 INFO:teuthology.orchestra.run.smithi076.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T22:43:55.567 INFO:teuthology.orchestra.run.smithi076.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T22:43:55.568 DEBUG:teuthology.orchestra.run.smithi076:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T22:43:55.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2024-09-14T22:44:02.092 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:44:02.131 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:44:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:44:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T22:44:02.490 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T22:44:02.491 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T22:44:03.403 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T22:44:03.404 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:44:03.404 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:44:03.404 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T22:44:03.404 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T22:44:03.404 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T22:44:03.404 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T22:44:03.404 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:44:03.404 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:44:03.730 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T22:44:03.813 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T22:44:03.814 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T22:44:03.814 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T22:44:03.814 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T22:44:03.814 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T22:44:03.814 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T22:44:03.814 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T22:44:03.814 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T22:44:03.873 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T22:44:03.873 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T22:44:03.873 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 82.el9 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8274670 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-09-14T22:44:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-09-14T22:44:03.914 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:44:03.915 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T22:44:03.915 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T22:44:03.915 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 5.14.0-505.el9.x86_64 2024-09-14T22:44:03.916 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T22:44:04.578 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-09-14T22:44:05.908 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T22:44:05.956 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-09-14T22:44:05.959 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-14T22:44:05.974 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.WbfKKwjhcZ 2024-09-14T22:44:05.974 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.WbfKKwjhcZ 2024-09-14T22:44:06.040 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.WbfKKwjhcZ 2024-09-14T22:44:06.170 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.WbfKKwjhcZ is 6KB 2024-09-14T22:44:06.219 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.WbfKKwjhcZ 2024-09-14T22:44:06.234 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T22:44:06.297 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T22:44:06.297 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T22:44:06.297 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T22:44:06.297 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T22:44:06.298 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T22:44:06.450 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-09-14T22:44:25.741 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T22:44:25.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-14T22:44:25.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:44:36.461 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T22:44:36.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-14T22:44:36.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:44:44.225 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-09-14T22:44:53.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-14T22:44:53.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:44:54.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-14T22:44:56.289 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-09-14T22:45:03.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-14T22:45:03.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:45:07.037 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-14T22:45:08.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-14T22:45:08.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:45:11.357 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-09-14T22:45:19.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-14T22:45:19.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:45:26.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-14T22:45:26.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:45:50.872 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-14T22:45:51.461 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-14T22:45:51.461 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-14T22:45:51.462 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-14T22:45:51.504 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-505.el9.x86_64 2024-09-14T22:45:51.504 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T22:45:51.504 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T22:45:51.505 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T22:45:52.506 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T22:45:52.506 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T22:45:52.575 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-09-14T22:45:52.596 DEBUG:teuthology.parallel:result is None 2024-09-14T22:45:58.280 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-09-14T22:45:58.652 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-14T22:45:58.652 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-14T22:45:58.653 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-09-14T22:45:58.670 INFO:teuthology.orchestra.run.smithi076.stdout:5.14.0-505.el9.x86_64 2024-09-14T22:45:58.670 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T22:45:58.670 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T22:45:58.670 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T22:45:59.672 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T22:45:59.672 DEBUG:teuthology.orchestra.run.smithi076:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T22:45:59.745 INFO:teuthology.orchestra.run.smithi076.stdout:ttyS1 2024-09-14T22:45:59.765 DEBUG:teuthology.parallel:result is None 2024-09-14T22:45:59.765 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T22:45:59.776 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T22:45:59.776 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T22:45:59.779 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T22:45:59.797 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T22:45:59.870 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T22:45:59.908 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T22:45:59.908 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T22:45:59.911 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T22:45:59.946 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T22:46:00.076 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T22:46:00.076 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T22:46:00.078 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T22:46:00.113 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T22:46:00.113 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T22:46:00.124 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T22:46:00.125 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T22:46:00.127 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T22:46:00.134 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T22:46:00.134 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T22:46:00.170 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T22:46:00.197 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T22:46:00.206 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T22:46:00.206 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T22:46:00.239 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T22:46:00.255 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T22:46:00.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T22:46:00.387 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T22:46:00.470 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T22:46:00.511 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:46:00.511 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T22:46:00.577 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T22:46:00.622 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T22:46:00.669 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T22:46:00.757 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T22:46:00.796 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:46:00.797 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T22:46:00.864 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-09-14T22:46:00.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-09-14T22:46:00.919 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T22:46:00.967 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T22:46:01.447 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T22:46:01.453 INFO:teuthology.task.internal:Starting timer... 2024-09-14T22:46:01.453 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T22:46:01.495 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T22:46:01.526 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-14T22:46:01.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-09-14T22:46:01.587 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-09-14T22:46:01.589 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-09-14T22:46:01.644 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-09-14T22:46:01.646 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T22:46:01.646 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-09-14T22:46:01.682 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-09-14T22:46:01.682 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-09-14T22:46:01.719 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-09-14T22:46:01.719 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi114.front.sepia.ceph.com': 'permissive'} 2024-09-14T22:46:01.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-14T22:46:01.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:46:01.753 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-14T22:46:01.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:46:01.788 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T22:46:01.788 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-09-14T22:46:01.825 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-09-14T22:46:01.861 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T22:46:01.875 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T22:46:01.886 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T22:46:01.887 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi076.front.sepia.ceph.com,smithi114.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T22:49:46.120 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi114.front.sepia.ceph.com')] 2024-09-14T22:49:46.122 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-14T22:49:46.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:49:46.202 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-09-14T22:49:46.280 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-14T22:49:46.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-14T22:49:46.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T22:49:46.359 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-14T22:49:46.436 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-14T22:49:46.436 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T22:49:46.447 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T22:49:46.448 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T22:49:46.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T22:49:46.450 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T22:49:46.451 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T22:49:46.478 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T22:49:46.491 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T22:49:46.507 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T22:49:46.515 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-09-14T22:49:46.521 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T22:49:46.526 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-09-14T22:49:46.540 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T22:49:46.545 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-09-14T22:49:46.555 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T22:49:46.556 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-09-14T22:49:46.570 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T22:49:46.584 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T22:49:46.617 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-09-14T22:49:46.619 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T22:49:46.620 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-09-14T22:49:46.620 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:49:46.620 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:49:46.620 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:49:46.620 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:49:46.647 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-14T22:49:46.651 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T22:49:46.651 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-14T22:49:46.651 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:49:46.651 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:49:46.651 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:49:46.651 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:49:46.652 INFO:teuthology.run_tasks:Running task install... 2024-09-14T22:49:46.660 DEBUG:teuthology.task.install:project ceph 2024-09-14T22:49:46.660 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-14T22:49:46.660 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'} 2024-09-14T22:49:46.660 INFO:teuthology.task.install:Using flavor: default 2024-09-14T22:49:46.669 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T22:49:46.669 INFO:teuthology.task.install:extra packages: [] 2024-09-14T22:49:46.670 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-14T22:49:46.670 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:49:46.672 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'tag': None, 'wait_for_package': False} 2024-09-14T22:49:46.673 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:49:46.823 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-14T22:49:46.823 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-14T22:49:46.827 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/ 2024-09-14T22:49:46.827 INFO:teuthology.task.install.rpm:Package version is 19.1.1-324.gecab7be5 2024-09-14T22:49:46.938 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T22:49:46.938 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:49:46.939 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T22:49:46.943 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/ecab7be58963a5bd03910cd73eef66405d7fdd50/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T22:49:46.943 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:49:46.943 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T22:49:46.965 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T22:49:46.965 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T22:49:46.968 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T22:49:46.969 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ecab7be58963a5bd03910cd73eef66405d7fdd50/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T22:49:47.030 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T22:49:47.033 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T22:49:47.105 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T22:49:47.111 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T22:49:47.169 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-09-14T22:49:47.171 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-09-14T22:49:47.175 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-09-14T22:49:47.177 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-09-14T22:49:47.399 INFO:teuthology.orchestra.run.smithi076.stdout:45 files removed 2024-09-14T22:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout:45 files removed 2024-09-14T22:49:47.426 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw 2024-09-14T22:49:47.434 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw 2024-09-14T22:49:48.213 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 166 kB/s | 84 kB 00:00 2024-09-14T22:49:48.236 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 161 kB/s | 84 kB 00:00 2024-09-14T22:49:48.595 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-09-14T22:49:48.605 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-09-14T22:49:48.853 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-14T22:49:48.872 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-14T22:49:49.445 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-14T22:49:50.564 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-14T22:49:51.397 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-14T22:49:52.565 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-14T22:49:55.539 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 20 MB/s | 6.5 MB 00:00 2024-09-14T22:49:57.724 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.5 MB 00:01 2024-09-14T22:49:57.873 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-14T22:49:58.944 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-14T22:49:59.284 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 94 kB/s | 19 kB 00:00 2024-09-14T22:50:00.055 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-14T22:50:04.392 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-14T22:50:05.543 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-14T22:50:06.760 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:50:06.767 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:06.767 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:50:06.767 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:06.767 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:50:06.767 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-14T22:50:06.767 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-09-14T22:50:06.767 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-14T22:50:06.767 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T22:50:06.768 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-14T22:50:06.769 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout:Installing weak dependencies: 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout:Install 30 Packages 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 2 Packages 2024-09-14T22:50:06.770 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:06.771 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 60 M 2024-09-14T22:50:06.771 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:50:07.785 INFO:teuthology.orchestra.run.smithi114.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 9.4 MB/s | 5.4 MB 00:00 2024-09-14T22:50:07.852 INFO:teuthology.orchestra.run.smithi114.stdout:(2/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 377 kB/s | 25 kB 00:00 2024-09-14T22:50:07.911 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 11 M 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.4 M 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.2 M 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T22:50:07.926 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.4 M 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 21 M 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.1.1-324.gecab7be5.el9 ceph 25 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 752 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 507 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.1.1-324.gecab7be5.el9 ceph 5.3 M 2024-09-14T22:50:07.927 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 45 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.1.1-324.gecab7be5.el9 ceph 136 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.1.1-324.gecab7be5.el9 ceph 162 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.1.1-324.gecab7be5.el9 ceph 321 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 301 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.1.1-324.gecab7be5.el9 ceph 99 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T22:50:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T22:50:07.929 INFO:teuthology.orchestra.run.smithi076.stdout:Installing weak dependencies: 2024-09-14T22:50:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T22:50:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:07.929 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:50:07.929 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:07.929 INFO:teuthology.orchestra.run.smithi076.stdout:Install 30 Packages 2024-09-14T22:50:07.929 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 2 Packages 2024-09-14T22:50:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:07.930 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 60 M 2024-09-14T22:50:07.930 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:50:07.960 INFO:teuthology.orchestra.run.smithi114.stdout:(3/32): ceph-radosgw-19.1.1-324.gecab7be5.el9.x 14 MB/s | 11 MB 00:00 2024-09-14T22:50:08.079 INFO:teuthology.orchestra.run.smithi114.stdout:(4/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 3.2 MB/s | 752 kB 00:00 2024-09-14T22:50:08.144 INFO:teuthology.orchestra.run.smithi114.stdout:(5/32): libradosstriper1-19.1.1-324.gecab7be5.e 2.7 MB/s | 507 kB 00:00 2024-09-14T22:50:08.203 INFO:teuthology.orchestra.run.smithi114.stdout:(6/32): python3-ceph-argparse-19.1.1-324.gecab7 774 kB/s | 45 kB 00:00 2024-09-14T22:50:08.278 INFO:teuthology.orchestra.run.smithi114.stdout:(7/32): python3-ceph-common-19.1.1-324.gecab7be 1.8 MB/s | 136 kB 00:00 2024-09-14T22:50:08.388 INFO:teuthology.orchestra.run.smithi114.stdout:(8/32): librgw2-19.1.1-324.gecab7be5.el9.x86_64 17 MB/s | 5.3 MB 00:00 2024-09-14T22:50:08.412 INFO:teuthology.orchestra.run.smithi114.stdout:(9/32): python3-cephfs-19.1.1-324.gecab7be5.el9 1.2 MB/s | 162 kB 00:00 2024-09-14T22:50:08.437 INFO:teuthology.orchestra.run.smithi114.stdout:(10/32): python3-rados-19.1.1-324.gecab7be5.el9 6.3 MB/s | 321 kB 00:00 2024-09-14T22:50:08.462 INFO:teuthology.orchestra.run.smithi114.stdout:(11/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 5.9 MB/s | 301 kB 00:00 2024-09-14T22:50:08.663 INFO:teuthology.orchestra.run.smithi114.stdout:(12/32): ceph-common-19.1.1-324.gecab7be5.el9.x 15 MB/s | 21 MB 00:01 2024-09-14T22:50:08.680 INFO:teuthology.orchestra.run.smithi114.stdout:(13/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 409 kB/s | 99 kB 00:00 2024-09-14T22:50:08.788 INFO:teuthology.orchestra.run.smithi114.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-09-14T22:50:08.805 INFO:teuthology.orchestra.run.smithi114.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 312 kB/s | 107 kB 00:00 2024-09-14T22:50:08.820 INFO:teuthology.orchestra.run.smithi076.stdout:(1/32): ceph-base-19.1.1-324.gecab7be5.el9.x86_ 9.0 MB/s | 5.4 MB 00:00 2024-09-14T22:50:08.830 INFO:teuthology.orchestra.run.smithi114.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 959 kB/s | 160 kB 00:00 2024-09-14T22:50:08.881 INFO:teuthology.orchestra.run.smithi114.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 845 kB/s | 42 kB 00:00 2024-09-14T22:50:08.886 INFO:teuthology.orchestra.run.smithi076.stdout:(2/32): ceph-selinux-19.1.1-324.gecab7be5.el9.x 379 kB/s | 25 kB 00:00 2024-09-14T22:50:08.931 INFO:teuthology.orchestra.run.smithi114.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-14T22:50:08.981 INFO:teuthology.orchestra.run.smithi114.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 717 kB/s | 36 kB 00:00 2024-09-14T22:50:09.006 INFO:teuthology.orchestra.run.smithi114.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-14T22:50:09.037 INFO:teuthology.orchestra.run.smithi076.stdout:(3/32): ceph-radosgw-19.1.1-324.gecab7be5.el9.x 13 MB/s | 11 MB 00:00 2024-09-14T22:50:09.040 INFO:teuthology.orchestra.run.smithi114.stdout:(21/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-09-14T22:50:09.065 INFO:teuthology.orchestra.run.smithi114.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-14T22:50:09.070 INFO:teuthology.orchestra.run.smithi076.stdout:(4/32): libcephfs2-19.1.1-324.gecab7be5.el9.x86 4.0 MB/s | 752 kB 00:00 2024-09-14T22:50:09.095 INFO:teuthology.orchestra.run.smithi076.stdout:(5/32): libradosstriper1-19.1.1-324.gecab7be5.e 8.5 MB/s | 507 kB 00:00 2024-09-14T22:50:09.115 INFO:teuthology.orchestra.run.smithi114.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-14T22:50:09.141 INFO:teuthology.orchestra.run.smithi114.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-14T22:50:09.154 INFO:teuthology.orchestra.run.smithi076.stdout:(6/32): python3-ceph-argparse-19.1.1-324.gecab7 774 kB/s | 45 kB 00:00 2024-09-14T22:50:09.166 INFO:teuthology.orchestra.run.smithi114.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 301 kB/s | 26 kB 00:00 2024-09-14T22:50:09.191 INFO:teuthology.orchestra.run.smithi114.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-14T22:50:09.225 INFO:teuthology.orchestra.run.smithi114.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-14T22:50:09.238 INFO:teuthology.orchestra.run.smithi076.stdout:(7/32): python3-ceph-common-19.1.1-324.gecab7be 1.6 MB/s | 136 kB 00:00 2024-09-14T22:50:09.258 INFO:teuthology.orchestra.run.smithi114.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-14T22:50:09.326 INFO:teuthology.orchestra.run.smithi114.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-14T22:50:09.409 INFO:teuthology.orchestra.run.smithi114.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-14T22:50:09.471 INFO:teuthology.orchestra.run.smithi076.stdout:(8/32): ceph-common-19.1.1-324.gecab7be5.el9.x8 17 MB/s | 21 MB 00:01 2024-09-14T22:50:09.496 INFO:teuthology.orchestra.run.smithi076.stdout:(9/32): python3-cephfs-19.1.1-324.gecab7be5.el9 626 kB/s | 162 kB 00:00 2024-09-14T22:50:09.536 INFO:teuthology.orchestra.run.smithi114.stdout:(31/32): librados2-19.1.1-324.gecab7be5.el9.x86 12 MB/s | 3.4 MB 00:00 2024-09-14T22:50:09.563 INFO:teuthology.orchestra.run.smithi076.stdout:(10/32): librgw2-19.1.1-324.gecab7be5.el9.x86_6 11 MB/s | 5.3 MB 00:00 2024-09-14T22:50:09.589 INFO:teuthology.orchestra.run.smithi076.stdout:(11/32): python3-rados-19.1.1-324.gecab7be5.el9 2.7 MB/s | 321 kB 00:00 2024-09-14T22:50:09.593 INFO:teuthology.orchestra.run.smithi114.stdout:(32/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 12 MB/s | 3.2 MB 00:00 2024-09-14T22:50:09.599 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:50:09.599 INFO:teuthology.orchestra.run.smithi114.stdout:Total 21 MB/s | 60 MB 00:02 2024-09-14T22:50:09.606 INFO:teuthology.orchestra.run.smithi076.stdout:(12/32): python3-rgw-19.1.1-324.gecab7be5.el9.x 2.3 MB/s | 99 kB 00:00 2024-09-14T22:50:09.631 INFO:teuthology.orchestra.run.smithi076.stdout:(13/32): python3-rbd-19.1.1-324.gecab7be5.el9.x 2.2 MB/s | 301 kB 00:00 2024-09-14T22:50:09.701 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:50:09.737 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:50:09.737 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:50:09.748 INFO:teuthology.orchestra.run.smithi076.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-09-14T22:50:09.773 INFO:teuthology.orchestra.run.smithi076.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 581 kB/s | 107 kB 00:00 2024-09-14T22:50:09.807 INFO:teuthology.orchestra.run.smithi076.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 797 kB/s | 160 kB 00:00 2024-09-14T22:50:09.858 INFO:teuthology.orchestra.run.smithi076.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 846 kB/s | 42 kB 00:00 2024-09-14T22:50:09.883 INFO:teuthology.orchestra.run.smithi076.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-14T22:50:09.908 INFO:teuthology.orchestra.run.smithi076.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-14T22:50:09.933 INFO:teuthology.orchestra.run.smithi076.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-09-14T22:50:09.958 INFO:teuthology.orchestra.run.smithi076.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-14T22:50:09.984 INFO:teuthology.orchestra.run.smithi076.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-14T22:50:10.034 INFO:teuthology.orchestra.run.smithi076.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-14T22:50:10.059 INFO:teuthology.orchestra.run.smithi076.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-14T22:50:10.085 INFO:teuthology.orchestra.run.smithi076.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 303 kB/s | 26 kB 00:00 2024-09-14T22:50:10.101 INFO:teuthology.orchestra.run.smithi076.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-14T22:50:10.127 INFO:teuthology.orchestra.run.smithi076.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-14T22:50:10.177 INFO:teuthology.orchestra.run.smithi076.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-14T22:50:10.227 INFO:teuthology.orchestra.run.smithi076.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-14T22:50:10.303 INFO:teuthology.orchestra.run.smithi076.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-14T22:50:10.414 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:50:10.415 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:50:10.428 INFO:teuthology.orchestra.run.smithi076.stdout:(31/32): librados2-19.1.1-324.gecab7be5.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-14T22:50:10.478 INFO:teuthology.orchestra.run.smithi076.stdout:(32/32): librbd1-19.1.1-324.gecab7be5.el9.x86_6 13 MB/s | 3.2 MB 00:00 2024-09-14T22:50:10.482 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:50:10.483 INFO:teuthology.orchestra.run.smithi076.stdout:Total 24 MB/s | 60 MB 00:02 2024-09-14T22:50:10.582 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:50:10.617 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:50:10.618 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:50:11.290 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:50:11.291 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:50:11.891 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:50:11.961 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-14T22:50:11.998 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-14T22:50:12.023 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-14T22:50:12.046 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-14T22:50:12.071 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-14T22:50:12.223 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-14T22:50:12.281 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-14T22:50:12.623 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-14T22:50:12.733 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T22:50:12.797 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:50:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T22:50:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-14T22:50:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-14T22:50:12.917 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-14T22:50:12.943 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-14T22:50:12.950 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-14T22:50:12.973 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-14T22:50:13.006 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-14T22:50:13.014 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-14T22:50:13.035 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-14T22:50:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-14T22:50:13.157 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-14T22:50:13.179 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T22:50:13.243 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T22:50:13.252 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-14T22:50:13.277 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-14T22:50:13.316 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-14T22:50:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-14T22:50:13.429 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-14T22:50:13.470 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-14T22:50:13.502 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-14T22:50:13.530 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-14T22:50:13.556 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-14T22:50:13.737 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-14T22:50:13.785 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T22:50:13.869 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-14T22:50:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T22:50:13.912 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-14T22:50:13.965 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 8/34 2024-09-14T22:50:13.987 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-14T22:50:14.003 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-14T22:50:14.033 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-14T22:50:14.049 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 9/34 2024-09-14T22:50:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-14T22:50:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-14T22:50:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 11/34 2024-09-14T22:50:14.207 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-14T22:50:14.231 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T22:50:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T22:50:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-14T22:50:14.441 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-14T22:50:14.476 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-14T22:50:14.533 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-14T22:50:14.553 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-14T22:50:14.562 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-14T22:50:14.581 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-14T22:50:14.591 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-14T22:50:14.624 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-14T22:50:14.652 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-14T22:50:14.730 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-14T22:50:14.758 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-14T22:50:14.934 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-14T22:50:14.979 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-14T22:50:15.075 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 22/34 2024-09-14T22:50:15.108 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 23/34 2024-09-14T22:50:15.144 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-14T22:50:15.626 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-14T22:50:15.656 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-14T22:50:15.726 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 26/34 2024-09-14T22:50:15.796 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 27/34 2024-09-14T22:50:15.825 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 28/34 2024-09-14T22:50:17.061 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T22:50:17.113 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T22:50:17.707 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T22:50:17.741 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-14T22:50:17.835 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-14T22:50:17.835 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T22:50:17.835 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T22:50:17.836 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:17.876 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T22:50:18.143 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T22:50:18.180 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T22:50:18.753 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 29/34 2024-09-14T22:50:18.808 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-14T22:50:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 30/34 2024-09-14T22:50:18.884 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T22:50:18.884 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T22:50:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:18.918 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T22:50:27.538 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T22:50:27.539 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-09-14T22:50:27.539 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-09-14T22:50:27.539 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-09-14T22:50:27.539 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-09-14T22:50:27.539 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-09-14T22:50:27.539 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-09-14T22:50:27.539 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-09-14T22:50:27.539 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:28.551 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-14T22:50:28.577 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-14T22:50:28.577 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:28.577 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T22:50:28.577 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T22:50:28.577 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T22:50:28.578 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:28.579 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-14T22:50:28.663 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-14T22:50:28.663 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T22:50:29.068 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T22:50:29.068 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-09-14T22:50:29.068 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-09-14T22:50:29.068 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-09-14T22:50:29.068 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-09-14T22:50:29.069 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-09-14T22:50:29.069 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-09-14T22:50:29.069 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-09-14T22:50:29.069 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T22:50:29.877 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-14T22:50:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-14T22:50:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-14T22:50:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-14T22:50:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-14T22:50:29.878 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-14T22:50:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T22:50:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-14T22:50:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-14T22:50:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-14T22:50:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-14T22:50:29.879 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-14T22:50:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T22:50:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-14T22:50:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-14T22:50:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-14T22:50:29.880 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-14T22:50:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-14T22:50:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-14T22:50:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-14T22:50:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-14T22:50:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-14T22:50:29.881 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-14T22:50:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-14T22:50:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-14T22:50:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-14T22:50:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-14T22:50:29.882 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-14T22:50:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-14T22:50:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-14T22:50:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T22:50:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-14T22:50:29.883 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-14T22:50:30.079 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-14T22:50:30.108 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 32/34 2024-09-14T22:50:30.108 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:30.108 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T22:50:30.109 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T22:50:30.109 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T22:50:30.109 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:30.110 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-14T22:50:30.198 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-14T22:50:30.198 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.079 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T22:50:31.080 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T22:50:31.081 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T22:50:31.082 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:31.082 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:50:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T22:50:31.371 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 1/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 3/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 4/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 5/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 6/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 7/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x 8/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86 9/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 10/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 11/34 2024-09-14T22:50:31.372 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 12/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 13/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-14T22:50:31.373 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-14T22:50:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-14T22:50:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-14T22:50:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-14T22:50:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-14T22:50:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-14T22:50:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-14T22:50:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.1.1-324.gecab7be5.el9.x86_64 31/34 2024-09-14T22:50:31.374 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-14T22:50:31.375 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 33/34 2024-09-14T22:50:31.449 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-test 2024-09-14T22:50:32.014 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:28 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:50:32.147 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:50:32.148 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:32.148 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:50:32.148 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:32.148 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-09-14T22:50:32.149 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:32.150 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 49 M 2024-09-14T22:50:32.150 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 209 M 2024-09-14T22:50:32.150 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:50:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-14T22:50:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:32.448 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-09-14T22:50:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.448 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:50:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T22:50:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T22:50:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T22:50:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:32.452 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:50:32.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-test 2024-09-14T22:50:32.959 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 309 kB/s | 163 kB 00:00 2024-09-14T22:50:33.367 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 325 kB/s | 304 kB 00:00 2024-09-14T22:50:33.386 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:28 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:50:33.524 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.1.1-324.gecab7be5.el9 ceph 49 M 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.1.1-324.gecab7be5.el9 ceph 163 k 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T22:50:33.525 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:33.526 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:50:33.526 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:33.526 INFO:teuthology.orchestra.run.smithi076.stdout:Install 4 Packages 2024-09-14T22:50:33.526 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:33.526 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 49 M 2024-09-14T22:50:33.526 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 209 M 2024-09-14T22:50:33.527 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:50:33.576 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-09-14T22:50:33.845 INFO:teuthology.orchestra.run.smithi076.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-14T22:50:33.895 INFO:teuthology.orchestra.run.smithi076.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-14T22:50:34.045 INFO:teuthology.orchestra.run.smithi076.stdout:(3/4): libcephsqlite-19.1.1-324.gecab7be5.el9.x 408 kB/s | 163 kB 00:00 2024-09-14T22:50:36.426 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 12 MB/s | 49 MB 00:03 2024-09-14T22:50:36.428 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:50:36.428 INFO:teuthology.orchestra.run.smithi114.stdout:Total 11 MB/s | 49 MB 00:04 2024-09-14T22:50:36.455 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:50:36.476 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:50:36.476 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:50:36.662 INFO:teuthology.orchestra.run.smithi076.stdout:(4/4): ceph-test-19.1.1-324.gecab7be5.el9.x86_6 16 MB/s | 49 MB 00:03 2024-09-14T22:50:36.664 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:50:36.664 INFO:teuthology.orchestra.run.smithi076.stdout:Total 16 MB/s | 49 MB 00:03 2024-09-14T22:50:36.692 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:50:36.711 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:50:36.711 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:50:36.879 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:50:36.879 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:50:37.098 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:50:37.098 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:50:37.399 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:50:37.474 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T22:50:37.532 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T22:50:37.553 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-14T22:50:37.617 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:50:37.714 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T22:50:37.755 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T22:50:37.781 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-14T22:50:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-14T22:50:42.217 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-14T22:50:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 3/4 2024-09-14T22:50:42.570 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-14T22:50:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-14T22:50:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-14T22:50:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-14T22:50:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T22:50:44.861 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T22:50:44.861 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:44.861 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:50:44.861 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:44.862 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:44.862 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T22:50:44.862 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T22:50:44.862 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:44.862 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:50:44.976 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 4/4 2024-09-14T22:50:44.976 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 1/4 2024-09-14T22:50:44.976 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2/4 2024-09-14T22:50:44.976 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T22:50:45.163 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:45.284 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:50:45.565 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph 2024-09-14T22:50:45.731 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:41 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:50:45.867 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:50:45.870 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:45.870 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:50:45.870 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:45.870 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:50:45.870 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-14T22:50:45.870 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:50:45.870 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-14T22:50:45.870 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T22:50:45.871 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T22:50:45.872 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout:Install 37 Packages 2024-09-14T22:50:45.873 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:45.874 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 30 M 2024-09-14T22:50:45.874 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 108 M 2024-09-14T22:50:45.874 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:50:46.134 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:41 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:50:46.274 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:50:46.276 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:46.276 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:50:46.276 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:46.276 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:50:46.276 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.1.1-324.gecab7be5.el9 ceph 6.5 k 2024-09-14T22:50:46.276 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:50:46.276 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.1.1-324.gecab7be5.el9 ceph 2.4 M 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.1.1-324.gecab7be5.el9 ceph 1.5 M 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 249 k 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.1.1-324.gecab7be5.el9 ceph 4.7 M 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 17 M 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T22:50:46.277 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T22:50:46.278 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:50:46.279 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:46.280 INFO:teuthology.orchestra.run.smithi076.stdout:Install 37 Packages 2024-09-14T22:50:46.280 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:46.280 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 30 M 2024-09-14T22:50:46.280 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 108 M 2024-09-14T22:50:46.280 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:50:46.327 INFO:teuthology.orchestra.run.smithi114.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 55 kB/s | 6.5 kB 00:00 2024-09-14T22:50:46.494 INFO:teuthology.orchestra.run.smithi114.stdout:(2/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 5.3 MB/s | 1.5 MB 00:00 2024-09-14T22:50:46.561 INFO:teuthology.orchestra.run.smithi114.stdout:(3/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 6.8 MB/s | 2.4 MB 00:00 2024-09-14T22:50:46.636 INFO:teuthology.orchestra.run.smithi114.stdout:(4/37): ceph-mgr-modules-core-19.1.1-324.gecab7 3.3 MB/s | 249 kB 00:00 2024-09-14T22:50:46.737 INFO:teuthology.orchestra.run.smithi114.stdout:(5/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-14T22:50:46.815 INFO:teuthology.orchestra.run.smithi076.stdout:(1/37): ceph-19.1.1-324.gecab7be5.el9.x86_64.rp 47 kB/s | 6.5 kB 00:00 2024-09-14T22:50:47.053 INFO:teuthology.orchestra.run.smithi114.stdout:(6/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-14T22:50:47.079 INFO:teuthology.orchestra.run.smithi114.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 573 kB/s | 253 kB 00:00 2024-09-14T22:50:47.154 INFO:teuthology.orchestra.run.smithi114.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-14T22:50:47.229 INFO:teuthology.orchestra.run.smithi114.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.6 MB/s | 125 kB 00:00 2024-09-14T22:50:47.254 INFO:teuthology.orchestra.run.smithi114.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 531 kB/s | 106 kB 00:00 2024-09-14T22:50:47.281 INFO:teuthology.orchestra.run.smithi076.stdout:(2/37): ceph-mgr-19.1.1-324.gecab7be5.el9.x86_6 2.5 MB/s | 1.5 MB 00:00 2024-09-14T22:50:47.313 INFO:teuthology.orchestra.run.smithi114.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-14T22:50:47.348 INFO:teuthology.orchestra.run.smithi114.stdout:(12/37): python3-cryptography-36.0.1-4.el9.x86_ 2.0 MB/s | 1.2 MB 00:00 2024-09-14T22:50:47.423 INFO:teuthology.orchestra.run.smithi076.stdout:(3/37): ceph-mds-19.1.1-324.gecab7be5.el9.x86_6 3.2 MB/s | 2.4 MB 00:00 2024-09-14T22:50:47.455 INFO:teuthology.orchestra.run.smithi114.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 940 kB/s | 188 kB 00:00 2024-09-14T22:50:47.480 INFO:teuthology.orchestra.run.smithi114.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 260 kB/s | 35 kB 00:00 2024-09-14T22:50:47.490 INFO:teuthology.orchestra.run.smithi076.stdout:(4/37): ceph-mgr-modules-core-19.1.1-324.gecab7 3.7 MB/s | 249 kB 00:00 2024-09-14T22:50:47.505 INFO:teuthology.orchestra.run.smithi114.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-14T22:50:47.531 INFO:teuthology.orchestra.run.smithi114.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 791 kB/s | 172 kB 00:00 2024-09-14T22:50:47.556 INFO:teuthology.orchestra.run.smithi114.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-14T22:50:47.582 INFO:teuthology.orchestra.run.smithi114.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 505 kB/s | 51 kB 00:00 2024-09-14T22:50:47.623 INFO:teuthology.orchestra.run.smithi114.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-09-14T22:50:47.657 INFO:teuthology.orchestra.run.smithi114.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-14T22:50:47.658 INFO:teuthology.orchestra.run.smithi076.stdout:(5/37): ceph-mon-19.1.1-324.gecab7be5.el9.x86_6 5.6 MB/s | 4.7 MB 00:00 2024-09-14T22:50:47.682 INFO:teuthology.orchestra.run.smithi114.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-14T22:50:47.708 INFO:teuthology.orchestra.run.smithi114.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 428 kB/s | 11 kB 00:00 2024-09-14T22:50:47.733 INFO:teuthology.orchestra.run.smithi114.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 712 kB/s | 18 kB 00:00 2024-09-14T22:50:47.741 INFO:teuthology.orchestra.run.smithi076.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-14T22:50:47.758 INFO:teuthology.orchestra.run.smithi114.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-14T22:50:47.783 INFO:teuthology.orchestra.run.smithi114.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-14T22:50:47.791 INFO:teuthology.orchestra.run.smithi076.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-14T22:50:47.808 INFO:teuthology.orchestra.run.smithi114.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-09-14T22:50:47.829 INFO:teuthology.orchestra.run.smithi114.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 434 kB/s | 20 kB 00:00 2024-09-14T22:50:47.841 INFO:teuthology.orchestra.run.smithi076.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.7 MB/s | 135 kB 00:00 2024-09-14T22:50:47.854 INFO:teuthology.orchestra.run.smithi114.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.0 MB/s | 46 kB 00:00 2024-09-14T22:50:47.888 INFO:teuthology.orchestra.run.smithi114.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-14T22:50:47.891 INFO:teuthology.orchestra.run.smithi076.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-14T22:50:47.913 INFO:teuthology.orchestra.run.smithi114.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 940 kB/s | 79 kB 00:00 2024-09-14T22:50:47.925 INFO:teuthology.orchestra.run.smithi076.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 4.6 MB/s | 1.2 MB 00:00 2024-09-14T22:50:47.938 INFO:teuthology.orchestra.run.smithi114.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 325 kB/s | 16 kB 00:00 2024-09-14T22:50:47.950 INFO:teuthology.orchestra.run.smithi076.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-14T22:50:47.964 INFO:teuthology.orchestra.run.smithi114.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-14T22:50:47.989 INFO:teuthology.orchestra.run.smithi114.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-14T22:50:48.014 INFO:teuthology.orchestra.run.smithi114.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-14T22:50:48.039 INFO:teuthology.orchestra.run.smithi114.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-14T22:50:48.051 INFO:teuthology.orchestra.run.smithi076.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-14T22:50:48.065 INFO:teuthology.orchestra.run.smithi114.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-14T22:50:48.076 INFO:teuthology.orchestra.run.smithi076.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-14T22:50:48.090 INFO:teuthology.orchestra.run.smithi114.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-14T22:50:48.093 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:50:48.093 INFO:teuthology.orchestra.run.smithi114.stdout:Total 14 MB/s | 30 MB 00:02 2024-09-14T22:50:48.101 INFO:teuthology.orchestra.run.smithi076.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 692 kB/s | 35 kB 00:00 2024-09-14T22:50:48.143 INFO:teuthology.orchestra.run.smithi076.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.1 MB/s | 77 kB 00:00 2024-09-14T22:50:48.168 INFO:teuthology.orchestra.run.smithi076.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-09-14T22:50:48.194 INFO:teuthology.orchestra.run.smithi076.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-14T22:50:48.235 INFO:teuthology.orchestra.run.smithi076.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-14T22:50:48.245 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:50:48.253 INFO:teuthology.orchestra.run.smithi076.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.6 MB/s | 43 kB 00:00 2024-09-14T22:50:48.269 INFO:teuthology.orchestra.run.smithi076.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 221 kB/s | 22 kB 00:00 2024-09-14T22:50:48.285 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:50:48.285 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:50:48.303 INFO:teuthology.orchestra.run.smithi076.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-14T22:50:48.328 INFO:teuthology.orchestra.run.smithi076.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 462 kB/s | 11 kB 00:00 2024-09-14T22:50:48.354 INFO:teuthology.orchestra.run.smithi076.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 708 kB/s | 18 kB 00:00 2024-09-14T22:50:48.379 INFO:teuthology.orchestra.run.smithi076.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 929 kB/s | 23 kB 00:00 2024-09-14T22:50:48.404 INFO:teuthology.orchestra.run.smithi076.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 783 kB/s | 19 kB 00:00 2024-09-14T22:50:48.429 INFO:teuthology.orchestra.run.smithi076.stdout:(26/37): python3-jaraco-text-3.2.0-6.el9.noarch 797 kB/s | 20 kB 00:00 2024-09-14T22:50:48.455 INFO:teuthology.orchestra.run.smithi076.stdout:(27/37): python3-logutils-0.3.5-21.el9.noarch.r 1.8 MB/s | 46 kB 00:00 2024-09-14T22:50:48.480 INFO:teuthology.orchestra.run.smithi076.stdout:(28/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-09-14T22:50:48.505 INFO:teuthology.orchestra.run.smithi076.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-14T22:50:48.531 INFO:teuthology.orchestra.run.smithi076.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 662 kB/s | 16 kB 00:00 2024-09-14T22:50:48.560 INFO:teuthology.orchestra.run.smithi076.stdout:(31/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.1 MB/s | 90 kB 00:00 2024-09-14T22:50:48.585 INFO:teuthology.orchestra.run.smithi076.stdout:(32/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-14T22:50:48.610 INFO:teuthology.orchestra.run.smithi076.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-14T22:50:48.635 INFO:teuthology.orchestra.run.smithi076.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-14T22:50:48.661 INFO:teuthology.orchestra.run.smithi076.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 800 kB/s | 20 kB 00:00 2024-09-14T22:50:48.686 INFO:teuthology.orchestra.run.smithi076.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-09-14T22:50:48.787 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:50:48.787 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:50:48.878 INFO:teuthology.orchestra.run.smithi076.stdout:(37/37): ceph-osd-19.1.1-324.gecab7be5.el9.x86_ 11 MB/s | 17 MB 00:01 2024-09-14T22:50:48.882 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:50:48.882 INFO:teuthology.orchestra.run.smithi076.stdout:Total 12 MB/s | 30 MB 00:02 2024-09-14T22:50:49.034 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:50:49.074 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:50:49.074 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:50:49.554 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:50:49.554 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:50:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:50:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T22:50:50.235 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T22:50:50.262 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T22:50:50.334 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T22:50:50.378 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T22:50:50.411 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T22:50:50.520 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T22:50:50.598 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T22:50:50.638 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T22:50:50.673 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T22:50:50.703 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T22:50:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T22:50:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T22:50:50.811 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T22:50:50.854 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T22:50:50.889 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T22:50:50.927 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:50:50.937 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T22:50:50.969 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T22:50:50.998 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T22:50:51.043 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T22:50:51.050 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T22:50:51.083 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T22:50:51.153 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T22:50:51.183 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T22:50:51.199 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T22:50:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T22:50:51.261 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T22:50:51.322 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T22:50:51.323 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T22:50:51.370 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T22:50:51.401 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T22:50:51.426 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T22:50:51.451 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T22:50:51.482 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T22:50:51.486 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T22:50:51.516 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T22:50:51.551 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T22:50:51.584 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T22:50:51.633 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T22:50:51.668 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T22:50:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T22:50:51.702 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T22:50:51.723 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T22:50:51.751 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T22:50:51.783 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T22:50:51.802 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T22:50:51.865 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T22:50:51.891 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T22:50:51.939 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T22:50:52.010 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T22:50:52.057 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T22:50:52.099 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T22:50:52.152 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T22:50:52.191 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T22:50:52.219 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-14T22:50:52.241 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-14T22:50:52.243 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T22:50:52.267 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-14T22:50:52.267 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:52.267 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T22:50:52.267 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T22:50:52.267 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T22:50:52.267 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:52.299 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T22:50:52.484 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T22:50:52.532 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T22:50:52.609 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T22:50:52.707 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T22:50:52.748 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T22:50:52.863 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T22:50:53.019 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 32/37 2024-09-14T22:50:53.050 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-14T22:50:53.076 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 33/37 2024-09-14T22:50:53.076 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:53.076 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T22:50:53.080 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T22:50:53.080 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T22:50:53.080 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:53.692 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-14T22:50:53.719 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-14T22:50:53.720 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:53.720 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T22:50:53.720 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T22:50:53.720 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T22:50:53.720 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-14T22:50:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-14T22:50:54.175 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:54.175 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T22:50:54.175 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T22:50:54.175 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T22:50:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:54.401 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-14T22:50:54.425 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-14T22:50:54.425 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:54.426 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T22:50:54.426 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T22:50:54.426 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T22:50:54.426 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-14T22:50:54.534 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-14T22:50:54.562 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 34/37 2024-09-14T22:50:54.562 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:54.562 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T22:50:54.562 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T22:50:54.563 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T22:50:54.563 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:54.994 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-14T22:50:55.018 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 35/37 2024-09-14T22:50:55.018 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:55.018 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T22:50:55.018 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T22:50:55.018 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T22:50:55.018 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-14T22:50:55.268 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 36/37 2024-09-14T22:50:55.269 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:50:55.269 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T22:50:55.269 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T22:50:55.269 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T22:50:55.269 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:55.333 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-14T22:50:56.262 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-14T22:50:56.262 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-14T22:50:56.262 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-14T22:50:56.262 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T22:50:56.263 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T22:50:56.264 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T22:50:56.264 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T22:50:56.264 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T22:50:56.264 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T22:50:56.264 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T22:50:56.264 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T22:50:56.264 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T22:50:56.264 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T22:50:56.265 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T22:50:56.265 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T22:50:56.266 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T22:50:56.982 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 37/37 2024-09-14T22:50:56.982 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.1.1-324.gecab7be5.el9.x86_64 1/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 3/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 4/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 5/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.n 6/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T22:50:56.983 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T22:50:56.984 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T22:50:56.986 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T22:50:57.229 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T22:50:57.230 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T22:50:57.231 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T22:50:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T22:50:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T22:50:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:57.232 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:50:57.541 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-base 2024-09-14T22:50:58.061 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T22:50:58.061 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:58.061 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:50:58.061 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:58.061 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:58.061 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T22:50:58.062 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T22:50:58.063 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:58.064 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:50:58.112 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:54 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:50:58.206 INFO:teuthology.orchestra.run.smithi114.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:50:58.248 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:50:58.249 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:50:58.249 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:50:58.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install cephadm 2024-09-14T22:50:58.430 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-base 2024-09-14T22:50:58.879 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:54 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:50:59.000 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:53 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:50:59.010 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 765 k 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 771 k 2024-09-14T22:50:59.011 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:50:59.090 INFO:teuthology.orchestra.run.smithi076.stdout:Package ceph-base-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:50:59.131 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:50:59.132 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:50:59.132 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:50:59.207 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install cephadm 2024-09-14T22:50:59.275 INFO:teuthology.orchestra.run.smithi114.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-14T22:50:59.275 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:50:59.276 INFO:teuthology.orchestra.run.smithi114.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-14T22:50:59.276 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:50:59.281 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:50:59.281 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:50:59.287 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:50:59.287 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:50:59.362 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:50:59.767 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:54 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:50:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:50:59.898 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 765 k 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-09-14T22:50:59.899 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:50:59.900 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 765 k 2024-09-14T22:50:59.900 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 771 k 2024-09-14T22:50:59.900 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:50:59.954 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:00.152 INFO:teuthology.orchestra.run.smithi076.stdout:cephadm-19.1.1-324.gecab7be5.el9.noarch.rpm 3.0 MB/s | 765 kB 00:00 2024-09-14T22:51:00.152 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:00.153 INFO:teuthology.orchestra.run.smithi076.stdout:Total 2.9 MB/s | 765 kB 00:00 2024-09-14T22:51:00.153 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:00.158 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:00.158 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:00.164 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:00.164 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:00.238 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:00.593 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:00.594 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:00.594 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:00.594 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:00.594 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:00.594 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:00.713 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:00.805 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:00.829 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T22:51:01.177 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:01.401 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:57 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:01.410 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:01.410 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:01.537 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 145 k 2024-09-14T22:51:01.538 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 435 k 2024-09-14T22:51:01.539 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:01.664 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T22:51:01.747 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 695 kB/s | 145 kB 00:00 2024-09-14T22:51:01.748 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:01.748 INFO:teuthology.orchestra.run.smithi114.stdout:Total 690 kB/s | 145 kB 00:00 2024-09-14T22:51:01.748 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:01.755 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:01.756 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:01.813 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:01.813 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:01.957 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:02.032 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T22:51:02.056 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T22:51:02.056 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:51:02.056 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T22:51:02.056 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:02.239 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:57 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:02.376 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repo Size 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-324.gecab7be5.el9 ceph 145 k 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-09-14T22:51:02.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:02.378 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 145 k 2024-09-14T22:51:02.378 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 435 k 2024-09-14T22:51:02.378 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:02.566 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-immutable-object-cache-19.1.1-324.gecab7be 769 kB/s | 145 kB 00:00 2024-09-14T22:51:02.567 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:02.568 INFO:teuthology.orchestra.run.smithi076.stdout:Total 763 kB/s | 145 kB 00:00 2024-09-14T22:51:02.568 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:02.575 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:02.575 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:02.632 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:02.632 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:02.773 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:02.834 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T22:51:02.858 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T22:51:02.858 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:51:02.859 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T22:51:02.859 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T22:51:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:02.983 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:02.983 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:03.218 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr 2024-09-14T22:51:03.782 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.e 1/1 2024-09-14T22:51:03.782 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:03.782 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:03.783 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:03.783 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:03.783 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:03.790 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:59 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:03.881 INFO:teuthology.orchestra.run.smithi114.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:03.923 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:03.924 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:51:03.924 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:03.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T22:51:04.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-mgr 2024-09-14T22:51:04.563 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:00 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:04.621 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:59 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:04.696 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T22:51:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout:Install 12 Packages 2024-09-14T22:51:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:04.699 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 8.7 M 2024-09-14T22:51:04.699 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 104 M 2024-09-14T22:51:04.699 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:04.712 INFO:teuthology.orchestra.run.smithi076.stdout:Package ceph-mgr-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:04.753 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:04.754 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:51:04.755 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:04.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T22:51:05.368 INFO:teuthology.orchestra.run.smithi114.stdout:(1/12): ceph-prometheus-alerts-19.1.1-324.gecab 114 kB/s | 16 kB 00:00 2024-09-14T22:51:05.384 INFO:teuthology.orchestra.run.smithi114.stdout:(2/12): ceph-grafana-dashboards-19.1.1-324.geca 198 kB/s | 32 kB 00:00 2024-09-14T22:51:05.405 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:00 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:05.538 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:05.539 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:05.539 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:51:05.539 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:05.539 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:05.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 3.6 M 2024-09-14T22:51:05.539 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:51:05.539 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T22:51:05.539 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 32 k 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 16 k 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T22:51:05.540 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T22:51:05.541 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:05.541 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:05.541 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:05.541 INFO:teuthology.orchestra.run.smithi076.stdout:Install 12 Packages 2024-09-14T22:51:05.541 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:05.541 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 8.7 M 2024-09-14T22:51:05.541 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 104 M 2024-09-14T22:51:05.542 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:05.559 INFO:teuthology.orchestra.run.smithi114.stdout:(3/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 11 MB/s | 3.6 MB 00:00 2024-09-14T22:51:05.860 INFO:teuthology.orchestra.run.smithi114.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.8 MB/s | 863 kB 00:00 2024-09-14T22:51:05.910 INFO:teuthology.orchestra.run.smithi114.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-14T22:51:05.936 INFO:teuthology.orchestra.run.smithi114.stdout:(6/12): python3-protobuf-3.14.0-13.el9.noarch.r 489 kB/s | 269 kB 00:00 2024-09-14T22:51:05.961 INFO:teuthology.orchestra.run.smithi114.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-09-14T22:51:06.011 INFO:teuthology.orchestra.run.smithi114.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-09-14T22:51:06.053 INFO:teuthology.orchestra.run.smithi114.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 735 kB/s | 31 kB 00:00 2024-09-14T22:51:06.078 INFO:teuthology.orchestra.run.smithi114.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-14T22:51:06.079 INFO:teuthology.orchestra.run.smithi076.stdout:(1/12): ceph-prometheus-alerts-19.1.1-324.gecab 114 kB/s | 16 kB 00:00 2024-09-14T22:51:06.104 INFO:teuthology.orchestra.run.smithi076.stdout:(2/12): ceph-grafana-dashboards-19.1.1-324.geca 188 kB/s | 32 kB 00:00 2024-09-14T22:51:06.129 INFO:teuthology.orchestra.run.smithi114.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 11 MB/s | 2.0 MB 00:00 2024-09-14T22:51:06.162 INFO:teuthology.orchestra.run.smithi114.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2024-09-14T22:51:06.166 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:06.167 INFO:teuthology.orchestra.run.smithi114.stdout:Total 5.9 MB/s | 8.7 MB 00:01 2024-09-14T22:51:06.218 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:06.236 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:06.295 INFO:teuthology.orchestra.run.smithi076.stdout:(3/12): ceph-mgr-dashboard-19.1.1-324.gecab7be5 9.9 MB/s | 3.6 MB 00:00 2024-09-14T22:51:06.321 INFO:teuthology.orchestra.run.smithi076.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-14T22:51:06.371 INFO:teuthology.orchestra.run.smithi076.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-14T22:51:06.397 INFO:teuthology.orchestra.run.smithi076.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-14T22:51:06.430 INFO:teuthology.orchestra.run.smithi076.stdout:(7/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-14T22:51:06.448 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:06.448 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:06.480 INFO:teuthology.orchestra.run.smithi076.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-09-14T22:51:06.506 INFO:teuthology.orchestra.run.smithi076.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-14T22:51:06.547 INFO:teuthology.orchestra.run.smithi076.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 14 MB/s | 2.0 MB 00:00 2024-09-14T22:51:06.581 INFO:teuthology.orchestra.run.smithi076.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 3.0 MB/s | 863 kB 00:00 2024-09-14T22:51:06.606 INFO:teuthology.orchestra.run.smithi076.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.8 MB/s | 188 kB 00:00 2024-09-14T22:51:06.611 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:06.611 INFO:teuthology.orchestra.run.smithi076.stdout:Total 8.1 MB/s | 8.7 MB 00:01 2024-09-14T22:51:06.659 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:06.676 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:06.676 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:06.890 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:06.890 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:06.980 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:07.054 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T22:51:07.089 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T22:51:07.135 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T22:51:07.167 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T22:51:07.265 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T22:51:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T22:51:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T22:51:07.507 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T22:51:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T22:51:07.542 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T22:51:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T22:51:07.589 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T22:51:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-14T22:51:07.627 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T22:51:07.732 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T22:51:07.863 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T22:51:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T22:51:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T22:51:08.044 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T22:51:08.076 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 10/12 2024-09-14T22:51:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-14T22:51:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-14T22:51:09.134 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 11/12 2024-09-14T22:51:09.169 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-14T22:51:09.995 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T22:51:09.996 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T22:51:10.311 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T22:51:10.312 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T22:51:10.312 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T22:51:10.312 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T22:51:10.312 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T22:51:10.312 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T22:51:10.312 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:10.312 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:10.491 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 12/12 2024-09-14T22:51:10.491 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9 1/12 2024-09-14T22:51:10.491 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noar 2/12 2024-09-14T22:51:10.491 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9. 3/12 2024-09-14T22:51:10.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T22:51:10.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T22:51:10.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T22:51:10.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T22:51:10.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T22:51:10.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T22:51:10.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T22:51:10.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T22:51:10.539 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T22:51:10.805 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T22:51:10.806 INFO:teuthology.orchestra.run.smithi076.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T22:51:10.806 INFO:teuthology.orchestra.run.smithi076.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T22:51:10.806 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T22:51:10.806 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T22:51:10.806 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:10.806 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:11.045 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T22:51:11.101 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:07 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:11.234 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:11.235 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-14T22:51:11.235 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:51:11.235 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-14T22:51:11.235 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:11.235 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-14T22:51:11.235 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:11.236 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-14T22:51:11.237 INFO:teuthology.orchestra.run.smithi114.stdout:Install 12 Packages 2024-09-14T22:51:11.237 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:11.237 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 38 M 2024-09-14T22:51:11.237 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 212 M 2024-09-14T22:51:11.237 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:11.602 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:06 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:11.734 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout:====================================================================================== 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout:====================================================================================== 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 7.4 M 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T22:51:11.736 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout:====================================================================================== 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout:Install 12 Packages 2024-09-14T22:51:11.737 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:11.738 INFO:teuthology.orchestra.run.smithi114.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 972 kB/s | 191 kB 00:00 2024-09-14T22:51:11.738 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 38 M 2024-09-14T22:51:11.738 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 212 M 2024-09-14T22:51:11.739 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:11.770 INFO:teuthology.orchestra.run.smithi114.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.4 MB/s | 801 kB 00:00 2024-09-14T22:51:11.854 INFO:teuthology.orchestra.run.smithi114.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-14T22:51:11.904 INFO:teuthology.orchestra.run.smithi114.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 366 kB/s | 18 kB 00:00 2024-09-14T22:51:11.990 INFO:teuthology.orchestra.run.smithi076.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-14T22:51:11.996 INFO:teuthology.orchestra.run.smithi114.stdout:(5/12): ceph-mgr-diskprediction-local-19.1.1-32 16 MB/s | 7.4 MB 00:00 2024-09-14T22:51:12.021 INFO:teuthology.orchestra.run.smithi114.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-14T22:51:12.032 INFO:teuthology.orchestra.run.smithi076.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-14T22:51:12.057 INFO:teuthology.orchestra.run.smithi076.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-14T22:51:12.099 INFO:teuthology.orchestra.run.smithi076.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 437 kB/s | 18 kB 00:00 2024-09-14T22:51:12.138 INFO:teuthology.orchestra.run.smithi114.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-14T22:51:12.157 INFO:teuthology.orchestra.run.smithi076.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-09-14T22:51:12.183 INFO:teuthology.orchestra.run.smithi076.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 460 kB/s | 38 kB 00:00 2024-09-14T22:51:12.230 INFO:teuthology.orchestra.run.smithi114.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.5 MB/s | 3.0 MB 00:00 2024-09-14T22:51:12.283 INFO:teuthology.orchestra.run.smithi076.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 39 MB/s | 4.9 MB 00:00 2024-09-14T22:51:12.288 INFO:teuthology.orchestra.run.smithi114.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-09-14T22:51:12.308 INFO:teuthology.orchestra.run.smithi076.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-14T22:51:12.400 INFO:teuthology.orchestra.run.smithi076.stdout:(9/12): ceph-mgr-diskprediction-local-19.1.1-32 13 MB/s | 7.4 MB 00:00 2024-09-14T22:51:12.425 INFO:teuthology.orchestra.run.smithi076.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-14T22:51:12.455 INFO:teuthology.orchestra.run.smithi114.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-09-14T22:51:12.528 INFO:teuthology.orchestra.run.smithi076.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-09-14T22:51:12.539 INFO:teuthology.orchestra.run.smithi114.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 9.1 MB/s | 4.9 MB 00:00 2024-09-14T22:51:12.939 INFO:teuthology.orchestra.run.smithi114.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-14T22:51:12.940 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:12.941 INFO:teuthology.orchestra.run.smithi114.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-14T22:51:13.293 INFO:teuthology.orchestra.run.smithi076.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-14T22:51:13.293 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:13.294 INFO:teuthology.orchestra.run.smithi076.stdout:Total 25 MB/s | 38 MB 00:01 2024-09-14T22:51:13.294 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:13.320 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:13.320 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:13.648 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:13.673 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:13.674 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:13.753 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:13.754 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:14.089 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:14.089 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:14.531 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:14.588 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T22:51:14.637 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T22:51:14.738 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T22:51:14.780 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T22:51:14.846 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:14.922 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T22:51:14.963 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T22:51:14.976 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T22:51:14.987 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T22:51:15.055 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T22:51:15.073 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T22:51:15.117 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T22:51:15.302 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T22:51:15.333 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T22:51:15.412 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T22:51:15.717 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T22:51:15.827 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T22:51:16.060 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T22:51:16.189 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T22:51:17.057 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T22:51:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T22:51:18.098 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T22:51:18.130 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-14T22:51:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T22:51:18.542 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-14T22:51:20.627 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-14T22:51:20.627 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-14T22:51:20.627 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T22:51:20.627 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T22:51:20.627 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T22:51:20.627 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T22:51:20.627 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T22:51:20.627 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T22:51:20.628 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T22:51:20.628 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T22:51:20.628 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T22:51:20.628 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T22:51:21.022 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T22:51:21.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T22:51:21.023 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T22:51:21.024 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:21.024 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:21.114 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 12/12 2024-09-14T22:51:21.114 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7b 1/12 2024-09-14T22:51:21.114 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T22:51:21.114 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T22:51:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T22:51:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T22:51:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T22:51:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T22:51:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T22:51:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T22:51:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T22:51:21.115 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T22:51:21.414 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-rook 2024-09-14T22:51:21.535 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T22:51:21.536 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T22:51:21.537 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T22:51:21.537 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T22:51:21.537 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:21.537 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:21.927 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-mgr-rook 2024-09-14T22:51:21.989 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:17 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:22.122 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:22.123 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:22.123 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:51:22.123 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:22.123 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T22:51:22.124 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T22:51:22.125 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T22:51:22.125 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:22.125 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:22.125 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:22.125 INFO:teuthology.orchestra.run.smithi114.stdout:Install 13 Packages 2024-09-14T22:51:22.125 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:22.125 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 2.2 M 2024-09-14T22:51:22.126 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 26 M 2024-09-14T22:51:22.126 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:22.423 INFO:teuthology.orchestra.run.smithi114.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 190 kB/s | 19 kB 00:00 2024-09-14T22:51:22.448 INFO:teuthology.orchestra.run.smithi114.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 203 kB/s | 26 kB 00:00 2024-09-14T22:51:22.494 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:17 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:22.506 INFO:teuthology.orchestra.run.smithi114.stdout:(3/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 267 kB/s | 49 kB 00:00 2024-09-14T22:51:22.532 INFO:teuthology.orchestra.run.smithi114.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-14T22:51:22.557 INFO:teuthology.orchestra.run.smithi114.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-14T22:51:22.599 INFO:teuthology.orchestra.run.smithi114.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 772 kB/s | 32 kB 00:00 2024-09-14T22:51:22.624 INFO:teuthology.orchestra.run.smithi114.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 581 kB/s | 54 kB 00:00 2024-09-14T22:51:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 49 k 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T22:51:22.631 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout:Install 13 Packages 2024-09-14T22:51:22.632 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 2.2 M 2024-09-14T22:51:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 26 M 2024-09-14T22:51:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:22.650 INFO:teuthology.orchestra.run.smithi114.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 282 kB/s | 14 kB 00:00 2024-09-14T22:51:22.683 INFO:teuthology.orchestra.run.smithi114.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-14T22:51:22.725 INFO:teuthology.orchestra.run.smithi114.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-14T22:51:22.750 INFO:teuthology.orchestra.run.smithi114.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 904 kB/s | 59 kB 00:00 2024-09-14T22:51:22.775 INFO:teuthology.orchestra.run.smithi114.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-14T22:51:22.809 INFO:teuthology.orchestra.run.smithi114.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.7 MB/s | 1.0 MB 00:00 2024-09-14T22:51:22.812 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:22.812 INFO:teuthology.orchestra.run.smithi114.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-14T22:51:22.876 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:22.891 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:22.891 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:22.980 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:22.980 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 228 kB/s | 26 kB 00:00 2024-09-14T22:51:23.109 INFO:teuthology.orchestra.run.smithi076.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 139 kB/s | 19 kB 00:00 2024-09-14T22:51:23.135 INFO:teuthology.orchestra.run.smithi076.stdout:(3/13): ceph-mgr-rook-19.1.1-324.gecab7be5.el9. 303 kB/s | 49 kB 00:00 2024-09-14T22:51:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-14T22:51:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-14T22:51:23.310 INFO:teuthology.orchestra.run.smithi076.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 800 kB/s | 54 kB 00:00 2024-09-14T22:51:23.344 INFO:teuthology.orchestra.run.smithi076.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 427 kB/s | 32 kB 00:00 2024-09-14T22:51:23.369 INFO:teuthology.orchestra.run.smithi076.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.2 MB/s | 279 kB 00:00 2024-09-14T22:51:23.394 INFO:teuthology.orchestra.run.smithi076.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 172 kB/s | 14 kB 00:00 2024-09-14T22:51:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-14T22:51:23.453 INFO:teuthology.orchestra.run.smithi076.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-14T22:51:23.478 INFO:teuthology.orchestra.run.smithi076.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-14T22:51:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:23.521 INFO:teuthology.orchestra.run.smithi076.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-14T22:51:23.523 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:23.523 INFO:teuthology.orchestra.run.smithi076.stdout:Total 2.5 MB/s | 2.2 MB 00:00 2024-09-14T22:51:23.588 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:23.603 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:23.603 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:23.680 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T22:51:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:23.694 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:23.744 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T22:51:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T22:51:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T22:51:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T22:51:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T22:51:24.089 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T22:51:24.138 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T22:51:24.194 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:24.353 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T22:51:24.413 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T22:51:24.468 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T22:51:24.506 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T22:51:24.547 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T22:51:24.650 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T22:51:24.760 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T22:51:24.815 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T22:51:24.846 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T22:51:24.893 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T22:51:24.927 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T22:51:24.975 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T22:51:24.997 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-14T22:51:25.536 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T22:51:25.588 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T22:51:25.621 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T22:51:25.670 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T22:51:25.691 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-14T22:51:25.746 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-14T22:51:25.746 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T22:51:25.747 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T22:51:26.072 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T22:51:26.072 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:26.072 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T22:51:26.073 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T22:51:26.074 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T22:51:26.074 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T22:51:26.074 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T22:51:26.074 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:26.074 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:26.353 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T22:51:26.492 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 13/13 2024-09-14T22:51:26.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 1/13 2024-09-14T22:51:26.492 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T22:51:26.493 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T22:51:26.822 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T22:51:26.822 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:26.822 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:26.822 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:26.822 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T22:51:26.822 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:26.823 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:26.914 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:22 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:27.047 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:27.047 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:27.047 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:27.048 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:27.049 INFO:teuthology.orchestra.run.smithi114.stdout:Install 6 Packages 2024-09-14T22:51:27.049 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:27.049 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 7.0 M 2024-09-14T22:51:27.049 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 33 M 2024-09-14T22:51:27.049 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:27.160 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T22:51:27.403 INFO:teuthology.orchestra.run.smithi114.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.5 MB/s | 248 kB 00:00 2024-09-14T22:51:27.453 INFO:teuthology.orchestra.run.smithi114.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-14T22:51:27.478 INFO:teuthology.orchestra.run.smithi114.stdout:(3/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 609 kB/s | 146 kB 00:00 2024-09-14T22:51:27.503 INFO:teuthology.orchestra.run.smithi114.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-14T22:51:27.537 INFO:teuthology.orchestra.run.smithi114.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-14T22:51:27.620 INFO:teuthology.orchestra.run.smithi114.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-14T22:51:27.623 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:27.623 INFO:teuthology.orchestra.run.smithi114.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-14T22:51:27.694 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:27.705 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:27.705 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:27.724 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:22 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:27.804 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:27.805 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:27.858 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 146 k 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T22:51:27.859 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T22:51:27.860 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:27.860 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:27.860 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:27.860 INFO:teuthology.orchestra.run.smithi076.stdout:Install 6 Packages 2024-09-14T22:51:27.860 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:27.860 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 7.0 M 2024-09-14T22:51:27.860 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 33 M 2024-09-14T22:51:27.861 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T22:51:28.339 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T22:51:28.581 INFO:teuthology.orchestra.run.smithi076.stdout:(1/6): ceph-mgr-cephadm-19.1.1-324.gecab7be5.el 785 kB/s | 146 kB 00:00 2024-09-14T22:51:28.631 INFO:teuthology.orchestra.run.smithi076.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-14T22:51:28.657 INFO:teuthology.orchestra.run.smithi076.stdout:(3/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 948 kB/s | 248 kB 00:00 2024-09-14T22:51:28.682 INFO:teuthology.orchestra.run.smithi076.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-14T22:51:28.724 INFO:teuthology.orchestra.run.smithi076.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-14T22:51:28.861 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T22:51:28.883 INFO:teuthology.orchestra.run.smithi076.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-14T22:51:28.884 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:28.885 INFO:teuthology.orchestra.run.smithi076.stdout:Total 6.8 MB/s | 7.0 MB 00:01 2024-09-14T22:51:28.942 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T22:51:28.955 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:28.966 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:28.966 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:29.005 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T22:51:29.034 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-14T22:51:29.065 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:29.065 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:29.370 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:29.531 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T22:51:29.576 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T22:51:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-14T22:51:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-14T22:51:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T22:51:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T22:51:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T22:51:29.890 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T22:51:30.089 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T22:51:30.159 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:30.160 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:30.178 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T22:51:30.242 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T22:51:30.271 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-14T22:51:30.409 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-fuse 2024-09-14T22:51:30.974 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:26 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 6/6 2024-09-14T22:51:31.034 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 1/6 2024-09-14T22:51:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T22:51:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T22:51:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T22:51:31.035 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T22:51:31.107 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:31.108 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:51:31.108 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:31.108 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-14T22:51:31.108 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-14T22:51:31.108 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T22:51:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:31.109 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:31.109 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:31.109 INFO:teuthology.orchestra.run.smithi114.stdout:Install 2 Packages 2024-09-14T22:51:31.109 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:31.109 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 976 k 2024-09-14T22:51:31.109 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 2.9 M 2024-09-14T22:51:31.109 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:31.311 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:31.489 INFO:teuthology.orchestra.run.smithi114.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 643 kB/s | 80 kB 00:00 2024-09-14T22:51:31.599 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-fuse 2024-09-14T22:51:31.656 INFO:teuthology.orchestra.run.smithi114.stdout:(2/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 3.0 MB/s | 896 kB 00:00 2024-09-14T22:51:31.657 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:31.657 INFO:teuthology.orchestra.run.smithi114.stdout:Total 1.7 MB/s | 976 kB 00:00 2024-09-14T22:51:31.668 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:31.692 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:31.692 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:31.774 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:31.774 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:32.108 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T22:51:32.162 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:27 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:32.170 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-14T22:51:32.296 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:32.296 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 896 k 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout:Install 2 Packages 2024-09-14T22:51:32.297 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:32.298 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 976 k 2024-09-14T22:51:32.298 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 2.9 M 2024-09-14T22:51:32.298 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:32.451 INFO:teuthology.orchestra.run.smithi076.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 746 kB/s | 80 kB 00:00 2024-09-14T22:51:32.610 INFO:teuthology.orchestra.run.smithi076.stdout:(2/2): ceph-fuse-19.1.1-324.gecab7be5.el9.x86_6 3.3 MB/s | 896 kB 00:00 2024-09-14T22:51:32.610 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:32.610 INFO:teuthology.orchestra.run.smithi076.stdout:Total 3.0 MB/s | 976 kB 00:00 2024-09-14T22:51:32.622 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:32.644 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:32.644 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:32.724 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:32.725 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:32.831 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-14T22:51:32.831 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-14T22:51:32.914 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T22:51:33.109 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T22:51:33.109 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:33.109 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:33.109 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T22:51:33.109 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:33.109 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-14T22:51:33.400 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-volume 2024-09-14T22:51:33.804 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2/2 2024-09-14T22:51:33.804 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/2 2024-09-14T22:51:33.982 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:29 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:34.048 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T22:51:34.049 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:34.049 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:34.049 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T22:51:34.049 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:34.049 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:34.120 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:34.121 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-14T22:51:34.121 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:34.121 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-14T22:51:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:34.122 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:34.122 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:34.122 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-14T22:51:34.122 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:34.122 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 285 k 2024-09-14T22:51:34.122 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 1.3 M 2024-09-14T22:51:34.122 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:34.332 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-volume 2024-09-14T22:51:34.343 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 1.3 MB/s | 285 kB 00:00 2024-09-14T22:51:34.344 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:34.344 INFO:teuthology.orchestra.run.smithi114.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-14T22:51:34.344 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:34.352 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:34.352 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:34.382 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:34.383 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:34.605 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:34.670 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:34.686 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:34.687 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:51:34.687 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T22:51:34.687 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:34.915 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:29 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:35.048 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.1.1-324.gecab7be5.el9 ceph-noarch 285 k 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:35.049 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-09-14T22:51:35.050 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:35.050 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 285 k 2024-09-14T22:51:35.050 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 1.3 M 2024-09-14T22:51:35.050 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:35.281 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-volume-19.1.1-324.gecab7be5.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-14T22:51:35.282 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:35.282 INFO:teuthology.orchestra.run.smithi076.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-14T22:51:35.282 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:35.290 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:35.290 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:35.318 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:35.318 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:35.538 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:35.600 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:35.616 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:35.616 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:51:35.616 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T22:51:35.617 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:35.792 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:35.792 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:35.792 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:35.792 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:35.793 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:35.793 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:36.090 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librados-devel 2024-09-14T22:51:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 1/1 2024-09-14T22:51:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:36.644 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.1.1-324.gecab7be5.el9.noarch 2024-09-14T22:51:36.644 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:36.644 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:36.667 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:32 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:36.799 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:36.800 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:36.800 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-09-14T22:51:36.800 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:36.800 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:36.800 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-14T22:51:36.800 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:36.800 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:36.800 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:36.801 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-14T22:51:36.801 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:36.801 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 127 k 2024-09-14T22:51:36.801 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 456 k 2024-09-14T22:51:36.801 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:36.980 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install librados-devel 2024-09-14T22:51:37.015 INFO:teuthology.orchestra.run.smithi114.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 593 kB/s | 127 kB 00:00 2024-09-14T22:51:37.015 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:37.016 INFO:teuthology.orchestra.run.smithi114.stdout:Total 589 kB/s | 127 kB 00:00 2024-09-14T22:51:37.016 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:37.021 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:37.021 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:37.084 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:37.084 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:37.236 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:37.338 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:37.552 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:32 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:37.688 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repo Size 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 127 k 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 127 k 2024-09-14T22:51:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 456 k 2024-09-14T22:51:37.690 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:37.797 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:37.878 INFO:teuthology.orchestra.run.smithi076.stdout:librados-devel-19.1.1-324.gecab7be5.el9.x86_64. 673 kB/s | 127 kB 00:00 2024-09-14T22:51:37.879 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:37.879 INFO:teuthology.orchestra.run.smithi076.stdout:Total 669 kB/s | 127 kB 00:00 2024-09-14T22:51:37.879 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:37.885 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:37.885 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:37.948 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:37.948 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:38.045 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:38.045 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:38.045 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:38.045 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:38.045 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:38.045 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:38.097 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:38.200 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:38.302 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install libcephfs2 2024-09-14T22:51:38.661 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:38.883 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:34 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:38.918 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:38.918 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:38.918 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:38.918 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:38.918 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:38.918 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:38.974 INFO:teuthology.orchestra.run.smithi114.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:39.020 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:39.021 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:51:39.021 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:39.088 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install libcephfs-devel 2024-09-14T22:51:39.195 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install libcephfs2 2024-09-14T22:51:39.653 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:35 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:39.767 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:34 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:39.786 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-14T22:51:39.787 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:39.788 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 32 k 2024-09-14T22:51:39.788 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 142 k 2024-09-14T22:51:39.788 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:39.860 INFO:teuthology.orchestra.run.smithi076.stdout:Package libcephfs2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:39.905 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:39.906 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:51:39.906 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:39.943 INFO:teuthology.orchestra.run.smithi114.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 205 kB/s | 32 kB 00:00 2024-09-14T22:51:39.944 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:39.944 INFO:teuthology.orchestra.run.smithi114.stdout:Total 203 kB/s | 32 kB 00:00 2024-09-14T22:51:39.944 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:39.948 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:39.949 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:39.970 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:39.971 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:39.987 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install libcephfs-devel 2024-09-14T22:51:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:40.176 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:40.545 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:40.553 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:35 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:40.685 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:40.686 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repo Size 2024-09-14T22:51:40.686 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:40.686 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.1.1-324.gecab7be5.el9 ceph 32 k 2024-09-14T22:51:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:40.687 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:40.687 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:40.687 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-09-14T22:51:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:40.687 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 32 k 2024-09-14T22:51:40.687 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 142 k 2024-09-14T22:51:40.687 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:40.782 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:40.782 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:40.782 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:40.782 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:40.782 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:40.783 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:40.840 INFO:teuthology.orchestra.run.smithi076.stdout:libcephfs-devel-19.1.1-324.gecab7be5.el9.x86_64 208 kB/s | 32 kB 00:00 2024-09-14T22:51:40.841 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:40.841 INFO:teuthology.orchestra.run.smithi076.stdout:Total 206 kB/s | 32 kB 00:00 2024-09-14T22:51:40.841 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:40.844 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:40.844 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:40.863 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:40.863 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:40.953 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:40.976 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librados2 2024-09-14T22:51:41.070 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:41.443 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:41.544 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:37 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:41.636 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:41.667 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:41.667 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:41.679 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:41.680 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:51:41.680 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:41.749 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librbd1 2024-09-14T22:51:41.880 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install librados2 2024-09-14T22:51:42.309 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:38 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:42.400 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:42.437 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:37 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:42.443 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:42.444 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:51:42.444 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:42.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rados 2024-09-14T22:51:42.528 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:42.571 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:42.572 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:51:42.572 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:42.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install librbd1 2024-09-14T22:51:43.073 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:39 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:43.163 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:43.205 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:43.206 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:51:43.206 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:43.212 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:38 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:43.270 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rgw 2024-09-14T22:51:43.306 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:43.349 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:43.350 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:51:43.350 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:43.436 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install python3-rados 2024-09-14T22:51:43.834 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:39 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:43.924 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:43.967 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:43.968 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:51:43.968 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:44.001 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:38 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:44.040 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-cephfs 2024-09-14T22:51:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:Package python3-rados-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:44.134 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:44.135 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:51:44.135 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:44.220 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install python3-rgw 2024-09-14T22:51:44.606 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:40 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:44.696 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:44.739 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:44.740 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:51:44.740 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:44.785 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:39 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:44.805 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rbd 2024-09-14T22:51:44.877 INFO:teuthology.orchestra.run.smithi076.stdout:Package python3-rgw-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:44.920 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:44.921 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:51:44.921 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:45.004 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install python3-cephfs 2024-09-14T22:51:45.370 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:41 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:45.461 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:45.504 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:45.505 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-14T22:51:45.505 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:45.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-fuse 2024-09-14T22:51:45.579 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:40 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:45.676 INFO:teuthology.orchestra.run.smithi076.stdout:Package python3-cephfs-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:45.723 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:45.723 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:51:45.724 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:45.822 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install python3-rbd 2024-09-14T22:51:46.137 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:42 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:46.271 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:46.272 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-14T22:51:46.273 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:46.273 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 85 k 2024-09-14T22:51:46.273 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 231 k 2024-09-14T22:51:46.273 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:46.415 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:41 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:46.513 INFO:teuthology.orchestra.run.smithi076.stdout:Package python3-rbd-2:19.1.1-324.gecab7be5.el9.x86_64 is already installed. 2024-09-14T22:51:46.555 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:46.556 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2024-09-14T22:51:46.556 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:46.637 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install rbd-fuse 2024-09-14T22:51:46.730 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 186 kB/s | 85 kB 00:00 2024-09-14T22:51:46.730 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:46.730 INFO:teuthology.orchestra.run.smithi114.stdout:Total 186 kB/s | 85 kB 00:00 2024-09-14T22:51:46.730 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:46.737 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:46.737 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:46.802 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:46.802 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:46.958 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:47.060 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:47.201 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:42 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:47.334 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:47.334 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.1.1-324.gecab7be5.el9 ceph 85 k 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-09-14T22:51:47.335 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:47.336 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 85 k 2024-09-14T22:51:47.336 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 231 k 2024-09-14T22:51:47.336 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:47.426 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:47.519 INFO:teuthology.orchestra.run.smithi076.stdout:rbd-fuse-19.1.1-324.gecab7be5.el9.x86_64.rpm 465 kB/s | 85 kB 00:00 2024-09-14T22:51:47.519 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:47.520 INFO:teuthology.orchestra.run.smithi076.stdout:Total 462 kB/s | 85 kB 00:00 2024-09-14T22:51:47.520 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:47.527 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:47.527 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:47.589 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:47.590 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:47.669 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:47.669 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:47.670 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:47.670 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:47.670 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:47.757 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:47.867 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:47.973 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-mirror 2024-09-14T22:51:48.327 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:48.561 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:48.611 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:48.611 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:48.611 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:48.611 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:48.611 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:48.611 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:48.703 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:48.704 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-14T22:51:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:48.704 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 3.1 M 2024-09-14T22:51:48.704 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 13 M 2024-09-14T22:51:48.704 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:48.880 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install rbd-mirror 2024-09-14T22:51:49.462 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repo Size 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.1.1-324.gecab7be5.el9 ceph 3.1 M 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-09-14T22:51:49.602 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:49.603 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 3.1 M 2024-09-14T22:51:49.603 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 13 M 2024-09-14T22:51:49.603 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:50.209 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 2.1 MB/s | 3.1 MB 00:01 2024-09-14T22:51:50.209 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:50.209 INFO:teuthology.orchestra.run.smithi114.stdout:Total 2.1 MB/s | 3.1 MB 00:01 2024-09-14T22:51:50.210 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:50.219 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:50.219 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:50.300 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:50.300 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:50.348 INFO:teuthology.orchestra.run.smithi076.stdout:rbd-mirror-19.1.1-324.gecab7be5.el9.x86_64.rpm 4.2 MB/s | 3.1 MB 00:00 2024-09-14T22:51:50.348 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:50.349 INFO:teuthology.orchestra.run.smithi076.stdout:Total 4.2 MB/s | 3.1 MB 00:00 2024-09-14T22:51:50.349 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:50.358 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:50.358 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:50.438 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:50.438 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:50.818 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:50.819 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:51:50.819 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T22:51:50.819 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T22:51:50.819 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T22:51:50.819 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:50.873 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:50.932 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:50.957 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:50.957 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T22:51:50.958 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T22:51:50.958 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T22:51:50.958 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T22:51:50.958 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:51.857 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:51.857 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:51.857 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:51.857 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:51.857 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:51.857 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:52.092 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:52.093 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:52.093 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:52.093 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:52.093 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:52.093 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:52.120 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-nbd 2024-09-14T22:51:52.359 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install rbd-nbd 2024-09-14T22:51:52.697 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:48 ago on Sat 14 Sep 2024 10:50:04 PM UTC. 2024-09-14T22:51:52.834 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-14T22:51:52.835 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:52.835 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-14T22:51:52.835 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:52.835 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-14T22:51:52.835 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-14T22:51:52.836 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:52.836 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-14T22:51:52.836 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-14T22:51:52.836 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-14T22:51:52.836 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:52.836 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 170 k 2024-09-14T22:51:52.836 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 494 k 2024-09-14T22:51:52.836 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-14T22:51:52.946 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:47 ago on Sat 14 Sep 2024 10:50:05 PM UTC. 2024-09-14T22:51:53.080 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-09-14T22:51:53.081 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:53.081 INFO:teuthology.orchestra.run.smithi076.stdout: Package Architecture Version Repository Size 2024-09-14T22:51:53.081 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:53.081 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-09-14T22:51:53.081 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.1.1-324.gecab7be5.el9 ceph 170 k 2024-09-14T22:51:53.082 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:53.082 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-09-14T22:51:53.082 INFO:teuthology.orchestra.run.smithi076.stdout:================================================================================ 2024-09-14T22:51:53.082 INFO:teuthology.orchestra.run.smithi076.stdout:Install 1 Package 2024-09-14T22:51:53.082 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:53.082 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 170 k 2024-09-14T22:51:53.082 INFO:teuthology.orchestra.run.smithi076.stdout:Installed size: 494 k 2024-09-14T22:51:53.082 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-09-14T22:51:53.370 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 319 kB/s | 170 kB 00:00 2024-09-14T22:51:53.370 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:53.371 INFO:teuthology.orchestra.run.smithi114.stdout:Total 318 kB/s | 170 kB 00:00 2024-09-14T22:51:53.371 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-14T22:51:53.379 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-14T22:51:53.379 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-14T22:51:53.444 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-14T22:51:53.444 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-14T22:51:53.596 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-14T22:51:53.693 INFO:teuthology.orchestra.run.smithi076.stdout:rbd-nbd-19.1.1-324.gecab7be5.el9.x86_64.rpm 279 kB/s | 170 kB 00:00 2024-09-14T22:51:53.694 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-09-14T22:51:53.695 INFO:teuthology.orchestra.run.smithi076.stdout:Total 278 kB/s | 170 kB 00:00 2024-09-14T22:51:53.695 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-09-14T22:51:53.700 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:53.703 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-09-14T22:51:53.703 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-09-14T22:51:53.768 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-09-14T22:51:53.768 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-09-14T22:51:53.923 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-09-14T22:51:54.015 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:54.378 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:54.378 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:54.378 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-14T22:51:54.378 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:54.378 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:54.378 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-14T22:51:54.425 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:54.622 DEBUG:teuthology.parallel:result is None 2024-09-14T22:51:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 1/1 2024-09-14T22:51:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:54.693 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-09-14T22:51:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.1.1-324.gecab7be5.el9.x86_64 2024-09-14T22:51:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:54.693 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-09-14T22:51:54.946 DEBUG:teuthology.parallel:result is None 2024-09-14T22:51:54.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:51:55.090 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T22:51:55.111 INFO:teuthology.orchestra.run.smithi076.stdout:19.1.1-324.gecab7be5.el9 2024-09-14T22:51:55.112 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-14T22:51:55.112 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-14T22:51:55.114 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ecab7be58963a5bd03910cd73eef66405d7fdd50 2024-09-14T22:51:55.249 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T22:51:55.274 INFO:teuthology.orchestra.run.smithi114.stdout:19.1.1-324.gecab7be5.el9 2024-09-14T22:51:55.275 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-324.gecab7be5.el9 2024-09-14T22:51:55.275 INFO:teuthology.task.install:The correct ceph version 19.1.1-324.gecab7be5 is installed. 2024-09-14T22:51:55.276 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T22:51:55.277 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:55.277 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T22:51:55.304 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:51:55.304 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T22:51:55.344 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T22:51:55.344 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:55.344 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T22:51:55.370 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T22:51:55.435 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:51:55.435 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T22:51:55.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T22:51:55.532 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T22:51:55.532 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:55.532 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T22:51:55.558 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T22:51:55.624 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:51:55.624 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T22:51:55.652 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T22:51:55.721 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T22:51:55.721 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:55.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T22:51:55.747 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T22:51:55.812 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:51:55.812 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T22:51:55.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T22:51:55.908 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T22:51:55.992 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T22:51:55.993 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /var/log/ceph 2024-09-14T22:51:55.996 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2024-09-14T22:51:56.023 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T22:51:56.023 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T22:51:56.061 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T22:51:56.089 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T22:51:56.090 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T22:51:56.126 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T22:51:56.164 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T22:51:56.165 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T22:51:56.165 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7904734', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4271, 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'sleep_before_teardown': 0, 'subset': '97/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ecab7be58963a5bd03910cd73eef66405d7fdd50/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50', 'targets': {'smithi076.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPekzKbvGGUg4/nEap++Kfd5a3dDcStKT+CG6xu7MaMOVJUPQQxYKVpUwlSOUSspG2pko0Ej8tu4GVzlu6u9Wc=', 'smithi114.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEUwbkj0Vp8J0W60rvdzKo/8OLZDKEN00V34UJ2WNb1ovrd07/d981wXeiidBokwOdhTnmv2YLAON9Khoa0lu38='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-14_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T22:51:56.165 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T22:51:56.190 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T22:51:56.211 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T22:51:56.245 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T22:51:56.277 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:56.278 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T22:51:56.324 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T22:51:56.324 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-09-14T22:51:56.379 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T22:51:56.388 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:51:56.388 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-14T22:51:56.388 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:51:56.389 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T22:51:56.389 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-14 22:51:51.208199412 +0000 2024-09-14T22:51:56.389 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-14 22:49:25.626616366 +0000 2024-09-14T22:51:56.389 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-14 22:49:25.626616366 +0000 2024-09-14T22:51:56.389 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-09-14 22:49:25.626616366 +0000 2024-09-14T22:51:56.389 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T22:51:56.443 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-14T22:51:56.443 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-14T22:51:56.443 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000335705 s, 1.5 MB/s 2024-09-14T22:51:56.444 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T22:51:56.500 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-09-14T22:51:56.555 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T22:51:56.555 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:51:56.556 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-14T22:51:56.556 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:51:56.556 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T22:51:56.556 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-14 22:51:51.208199412 +0000 2024-09-14T22:51:56.556 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-14 22:49:25.961606341 +0000 2024-09-14T22:51:56.556 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-14 22:49:25.961606341 +0000 2024-09-14T22:51:56.556 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-09-14 22:49:25.961606341 +0000 2024-09-14T22:51:56.556 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T22:51:56.618 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-14T22:51:56.618 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-14T22:51:56.618 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.0001897 s, 2.7 MB/s 2024-09-14T22:51:56.619 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T22:51:56.675 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-09-14T22:51:56.731 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T22:51:56.731 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:51:56.731 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-14T22:51:56.731 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:51:56.731 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T22:51:56.731 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-14 22:51:51.209199382 +0000 2024-09-14T22:51:56.731 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-14 22:49:26.456591527 +0000 2024-09-14T22:51:56.732 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-14 22:49:26.456591527 +0000 2024-09-14T22:51:56.732 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-09-14 22:49:26.456591527 +0000 2024-09-14T22:51:56.732 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T22:51:56.794 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-14T22:51:56.794 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-14T22:51:56.794 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000207708 s, 2.5 MB/s 2024-09-14T22:51:56.795 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T22:51:56.851 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-09-14T22:51:56.906 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T22:51:56.906 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:51:56.907 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-14T22:51:56.907 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:51:56.907 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T22:51:56.907 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-14 22:51:51.209199382 +0000 2024-09-14T22:51:56.907 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-14 22:49:26.787581621 +0000 2024-09-14T22:51:56.907 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-14 22:49:26.787581621 +0000 2024-09-14T22:51:56.907 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-09-14 22:49:26.787581621 +0000 2024-09-14T22:51:56.907 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T22:51:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-14T22:51:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-14T22:51:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.00019201 s, 2.7 MB/s 2024-09-14T22:51:56.970 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T22:51:57.026 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:51:57.026 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:51:57.026 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T22:51:57.045 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T22:51:57.046 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-09-14T22:51:57.104 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T22:51:57.104 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:51:57.104 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-14T22:51:57.104 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:51:57.104 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T22:51:57.104 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-14 22:51:51.073595528 +0000 2024-09-14T22:51:57.105 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-14 22:49:25.672850953 +0000 2024-09-14T22:51:57.105 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-14 22:49:25.672850953 +0000 2024-09-14T22:51:57.105 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-14 22:49:25.672850953 +0000 2024-09-14T22:51:57.105 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T22:51:57.172 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-14T22:51:57.172 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-14T22:51:57.172 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000259833 s, 2.0 MB/s 2024-09-14T22:51:57.173 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T22:51:57.233 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-14 22:51:51.074595507 +0000 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-14 22:49:25.997844051 +0000 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-14 22:49:25.997844051 +0000 2024-09-14T22:51:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-14 22:49:25.997844051 +0000 2024-09-14T22:51:57.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T22:51:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-14T22:51:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-14T22:51:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000220334 s, 2.3 MB/s 2024-09-14T22:51:57.360 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T22:51:57.417 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-09-14T22:51:57.473 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T22:51:57.474 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:51:57.474 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-14T22:51:57.474 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:51:57.474 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T22:51:57.474 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-14 22:51:51.074595507 +0000 2024-09-14T22:51:57.474 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-14 22:49:26.312837357 +0000 2024-09-14T22:51:57.474 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-14 22:49:26.312837357 +0000 2024-09-14T22:51:57.474 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-14 22:49:26.312837357 +0000 2024-09-14T22:51:57.475 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T22:51:57.539 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-14T22:51:57.539 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-14T22:51:57.539 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000263381 s, 1.9 MB/s 2024-09-14T22:51:57.540 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T22:51:57.598 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-09-14T22:51:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T22:51:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T22:51:57.654 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-14T22:51:57.654 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T22:51:57.654 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T22:51:57.654 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-14 22:51:51.074595507 +0000 2024-09-14T22:51:57.654 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-14 22:49:26.657830026 +0000 2024-09-14T22:51:57.654 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-14 22:49:26.657830026 +0000 2024-09-14T22:51:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-14 22:49:26.657830026 +0000 2024-09-14T22:51:57.654 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T22:51:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-14T22:51:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-14T22:51:57.717 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.00025572 s, 2.0 MB/s 2024-09-14T22:51:57.718 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T22:51:57.775 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:51:57.775 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T22:51:57.775 INFO:tasks.ceph:Generating config... 2024-09-14T22:51:57.778 INFO:tasks.ceph:[client] rbd cache = True 2024-09-14T22:51:57.778 INFO:tasks.ceph:[client] rbd cache policy = writeback 2024-09-14T22:51:57.778 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T22:51:57.778 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T22:51:57.778 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T22:51:57.778 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T22:51:57.778 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T22:51:57.778 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T22:51:57.778 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T22:51:57.778 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T22:51:57.778 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-14T22:51:57.778 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T22:51:57.779 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T22:51:57.780 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-14T22:51:57.780 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T22:51:57.780 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T22:51:57.831 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T22:51:57.834 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T22:51:57.879 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T22:51:57.905 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.76'), ('mon.c', '[v2:172.21.15.76:3301,v1:172.21.15.76:6790]'), ('mon.b', '172.21.15.114')] 2024-09-14T22:51:57.905 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790],172.21.15.114', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writeback'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T22:51:57.906 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:57.906 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T22:51:57.961 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.76 --addv c '[v2:172.21.15.76:3301,v1:172.21.15.76:6790]' --add b 172.21.15.114 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:51:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:51:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: generated fsid 9a42a40e-8d25-4801-9835-f823a294bdbe 2024-09-14T22:51:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:setting min_mon_release = quincy 2024-09-14T22:51:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:epoch 0 2024-09-14T22:51:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:fsid 9a42a40e-8d25-4801-9835-f823a294bdbe 2024-09-14T22:51:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:last_changed 2024-09-14T22:51:58.046902+0000 2024-09-14T22:51:58.048 INFO:teuthology.orchestra.run.smithi076.stdout:created 2024-09-14T22:51:58.046902+0000 2024-09-14T22:51:58.049 INFO:teuthology.orchestra.run.smithi076.stdout:min_mon_release 17 (quincy) 2024-09-14T22:51:58.049 INFO:teuthology.orchestra.run.smithi076.stdout:election_strategy: 1 2024-09-14T22:51:58.049 INFO:teuthology.orchestra.run.smithi076.stdout:0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-09-14T22:51:58.049 INFO:teuthology.orchestra.run.smithi076.stdout:1: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.b 2024-09-14T22:51:58.049 INFO:teuthology.orchestra.run.smithi076.stdout:2: [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] mon.c 2024-09-14T22:51:58.049 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T22:51:58.051 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T22:51:58.067 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9a42a40e-8d25-4801-9835-f823a294bdbe... 2024-09-14T22:51:58.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T22:51:58.110 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T22:51:58.151 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-09-14T22:51:58.151 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-09-14T22:51:58.151 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T22:51:58.151 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-09-14T22:51:58.151 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.151 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-09-14T22:51:58.152 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.153 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T22:51:58.153 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T22:51:58.153 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.153 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-09-14T22:51:58.153 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.154 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-09-14T22:51:58.154 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-09-14T22:51:58.154 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-09-14T22:51:58.154 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-09-14T22:51:58.154 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-09-14T22:51:58.154 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-09-14T22:51:58.155 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-09-14T22:51:58.155 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T22:51:58.155 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T22:51:58.155 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-09-14T22:51:58.155 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.156 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T22:51:58.156 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.156 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-09-14T22:51:58.156 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.156 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-09-14T22:51:58.156 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.157 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-09-14T22:51:58.157 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-09-14T22:51:58.157 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-09-14T22:51:58.157 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790],172.21.15.114" 2024-09-14T22:51:58.157 INFO:teuthology.orchestra.run.smithi076.stdout: mon client directed command retry = 5 2024-09-14T22:51:58.157 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on pool no app = False 2024-09-14T22:51:58.157 INFO:teuthology.orchestra.run.smithi076.stdout: ms inject socket failures = 5000 2024-09-14T22:51:58.158 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = 9a42a40e-8d25-4801-9835-f823a294bdbe 2024-09-14T22:51:58.158 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.158 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-09-14T22:51:58.158 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-09-14T22:51:58.158 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.158 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-09-14T22:51:58.158 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-09-14T22:51:58.159 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-09-14T22:51:58.159 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.159 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-09-14T22:51:58.159 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-09-14T22:51:58.159 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.159 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-09-14T22:51:58.159 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-09-14T22:51:58.160 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-09-14T22:51:58.160 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-09-14T22:51:58.160 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.160 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-09-14T22:51:58.160 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-09-14T22:51:58.160 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.161 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-09-14T22:51:58.161 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.161 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-09-14T22:51:58.161 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.161 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-09-14T22:51:58.161 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.162 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-09-14T22:51:58.162 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-09-14T22:51:58.162 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.162 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-09-14T22:51:58.162 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-09-14T22:51:58.162 INFO:teuthology.orchestra.run.smithi076.stdout: bluefs allocator = hybrid 2024-09-14T22:51:58.162 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore allocator = hybrid 2024-09-14T22:51:58.163 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-09-14T22:51:58.163 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-09-14T22:51:58.163 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 1/20 2024-09-14T22:51:58.163 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 1/20 2024-09-14T22:51:58.163 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-09-14T22:51:58.163 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-09-14T22:51:58.163 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 4/10 2024-09-14T22:51:58.163 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T22:51:58.164 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-09-14T22:51:58.164 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-09-14T22:51:58.164 INFO:teuthology.orchestra.run.smithi076.stdout: osd blocked scrub grace period = 3600 2024-09-14T22:51:58.164 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-09-14T22:51:58.164 INFO:teuthology.orchestra.run.smithi076.stdout: osd max markdown count = 1000 2024-09-14T22:51:58.164 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-09-14T22:51:58.164 INFO:teuthology.orchestra.run.smithi076.stdout: osd shutdown pgref assert = True 2024-09-14T22:51:58.165 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.165 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-09-14T22:51:58.165 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-09-14T22:51:58.165 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-09-14T22:51:58.165 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-09-14T22:51:58.165 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-09-14T22:51:58.166 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-09-14T22:51:58.166 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-09-14T22:51:58.166 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-09-14T22:51:58.166 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.166 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-09-14T22:51:58.166 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-09-14T22:51:58.166 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-09-14T22:51:58.167 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-09-14T22:51:58.167 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-09-14T22:51:58.167 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-09-14T22:51:58.167 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-09-14T22:51:58.167 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-09-14T22:51:58.167 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-09-14T22:51:58.168 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-09-14T22:51:58.168 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-09-14T22:51:58.168 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.168 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T22:51:58.168 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T22:51:58.169 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T22:51:58.169 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.169 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T22:51:58.169 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T22:51:58.169 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T22:51:58.169 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.170 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-09-14T22:51:58.170 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-09-14T22:51:58.170 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.170 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-09-14T22:51:58.170 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:51:58.170 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-09-14T22:51:58.171 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-09-14T22:51:58.171 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-09-14T22:51:58.171 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-09-14T22:51:58.171 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T22:51:58.171 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T22:51:58.171 INFO:teuthology.orchestra.run.smithi076.stdout: rbd cache = True 2024-09-14T22:51:58.171 INFO:teuthology.orchestra.run.smithi076.stdout: rbd cache policy = writeback 2024-09-14T22:51:58.171 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-09-14T22:51:58.172 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-09-14T22:51:58.172 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-09-14T22:51:58.174 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-09-14T22:51:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-09-14T22:51:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T22:51:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-09-14T22:51:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-09-14T22:51:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-09-14T22:51:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-09-14T22:51:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-09-14T22:51:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-09-14T22:51:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-09-14T22:51:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T22:51:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-09-14T22:51:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T22:51:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T22:51:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-09-14T22:51:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-09-14T22:51:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T22:51:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790],172.21.15.114" 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: mon client directed command retry = 5 2024-09-14T22:51:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on pool no app = False 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: ms inject socket failures = 5000 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = 9a42a40e-8d25-4801-9835-f823a294bdbe 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 600 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-09-14T22:51:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug trim objects = true 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-09-14T22:51:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-09-14T22:51:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-09-14T22:51:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-09-14T22:51:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-09-14T22:51:58.183 INFO:teuthology.orchestra.run.smithi114.stdout: bluefs allocator = hybrid 2024-09-14T22:51:58.183 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore allocator = hybrid 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 1/20 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 1/20 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 4/10 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: osd blocked scrub grace period = 3600 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-09-14T22:51:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: osd max markdown count = 1000 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: osd shutdown pgref assert = True 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.185 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T22:51:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-09-14T22:51:58.188 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-14T22:51:58.188 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-09-14T22:51:58.188 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-09-14T22:51:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-09-14T22:51:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-09-14T22:51:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T22:51:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T22:51:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: rbd cache = True 2024-09-14T22:51:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: rbd cache policy = writeback 2024-09-14T22:51:58.190 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-09-14T22:51:58.190 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-09-14T22:51:58.190 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-09-14T22:51:58.190 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T22:51:58.191 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T22:51:58.270 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T22:51:58.270 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:58.271 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T22:51:58.287 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:58.287 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T22:51:58.343 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:51:58.344 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:58.344 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T22:51:58.344 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T22:51:58.418 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:51:58.418 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:51:58.474 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:51:58.474 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:51:58.474 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T22:51:58.475 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T22:51:58.510 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:51:58.511 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:51:58.566 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T22:51:58.567 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T22:51:58.567 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T22:51:58.618 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T22:51:58.622 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T22:51:58.691 INFO:teuthology.orchestra.run.smithi114.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T22:51:58.693 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T22:51:58.694 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T22:51:58.694 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T22:51:58.734 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T22:51:58.747 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-14T22:51:58.789 INFO:teuthology.orchestra.run.smithi114.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-14T22:51:58.801 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T22:51:58.801 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T22:51:58.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T22:51:58.826 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:51:58.826 INFO:tasks.ceph:role: osd.0 2024-09-14T22:51:58.826 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:51:58.826 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T22:51:58.898 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:51:58.899 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:51:59.399 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-09-14T22:51:59.401 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-09-14T22:51:59.401 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T22:51:59.507 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T22:51:59.538 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T22:51:59.604 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:51:59.604 INFO:tasks.ceph:role: osd.1 2024-09-14T22:51:59.604 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:51:59.604 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T22:51:59.676 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:51:59.676 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:51:59.676 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:51:59.676 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T22:51:59.676 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:51:59.676 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:51:59.677 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:51:59.677 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T22:51:59.677 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:51:59.677 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:52:00.129 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-09-14T22:52:00.131 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-09-14T22:52:00.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T22:52:00.204 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T22:52:00.274 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T22:52:00.339 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:52:00.339 INFO:tasks.ceph:role: osd.2 2024-09-14T22:52:00.339 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:52:00.339 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T22:52:00.411 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:52:00.412 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:52:00.898 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-09-14T22:52:00.900 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-09-14T22:52:00.900 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T22:52:00.993 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T22:52:01.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T22:52:01.089 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T22:52:01.089 INFO:tasks.ceph:role: osd.3 2024-09-14T22:52:01.089 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:52:01.089 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T22:52:01.162 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:52:01.162 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:52:01.162 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:52:01.162 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T22:52:01.162 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:52:01.162 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:52:01.162 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:52:01.163 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T22:52:01.163 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:52:01.163 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:52:01.678 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-09-14T22:52:01.680 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-09-14T22:52:01.681 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-14T22:52:01.790 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T22:52:01.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:01.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:01.921+0000 7f5301e0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T22:52:01.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:01.925+0000 7f5301e0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T22:52:01.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:01.925+0000 7f5301e0c640 -1 bdev(0x55f760029800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T22:52:01.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:01.925+0000 7f5301e0c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T22:52:04.831 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T22:52:04.857 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:04.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:04.941+0000 7f3fcb52e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T22:52:04.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:04.941+0000 7f3fcb52e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T22:52:04.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:04.941+0000 7f3fcb52e640 -1 bdev(0x55ff83d09800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T22:52:04.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:04.941+0000 7f3fcb52e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T22:52:07.860 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T22:52:07.886 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:07.970+0000 7fb677633640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T22:52:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:07.970+0000 7fb677633640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T22:52:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:07.970+0000 7fb677633640 -1 bdev(0x560a9ef01800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T22:52:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:07.970+0000 7fb677633640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T22:52:11.128 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T22:52:11.155 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:11.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:11.241+0000 7f0a1c631640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T22:52:11.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:11.241+0000 7f0a1c631640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T22:52:11.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:11.241+0000 7f0a1c631640 -1 bdev(0x55e6147bb800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T22:52:11.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:52:11.241+0000 7f0a1c631640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T22:52:14.153 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T22:52:14.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T22:52:14.212 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:52:14.212 INFO:tasks.ceph:role: osd.4 2024-09-14T22:52:14.212 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:52:14.213 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T22:52:14.289 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:52:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:52:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:52:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T22:52:14.289 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:52:14.289 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:52:14.289 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:52:14.289 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T22:52:14.290 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:52:14.290 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:52:14.796 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-14T22:52:14.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-14T22:52:14.798 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-14T22:52:14.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T22:52:14.987 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T22:52:15.056 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:52:15.056 INFO:tasks.ceph:role: osd.5 2024-09-14T22:52:15.056 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:52:15.057 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T22:52:15.131 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:52:15.132 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:52:15.573 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-14T22:52:15.575 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-14T22:52:15.575 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-14T22:52:15.667 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T22:52:15.696 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-14T22:52:15.762 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:52:15.762 INFO:tasks.ceph:role: osd.6 2024-09-14T22:52:15.762 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:52:15.762 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:52:15.835 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:52:16.321 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-14T22:52:16.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-14T22:52:16.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-14T22:52:16.465 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-14T22:52:16.495 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-14T22:52:16.560 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T22:52:16.561 INFO:tasks.ceph:role: osd.7 2024-09-14T22:52:16.561 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:52:16.561 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T22:52:16.632 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T22:52:16.632 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T22:52:16.632 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T22:52:16.633 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T22:52:16.633 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T22:52:16.633 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-14T22:52:16.633 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T22:52:16.633 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T22:52:16.633 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T22:52:16.633 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T22:52:17.151 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-14T22:52:17.153 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-14T22:52:17.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-14T22:52:17.301 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-14T22:52:17.334 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:17.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:17.436+0000 7f197aa0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T22:52:17.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:17.436+0000 7f197aa0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T22:52:17.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:17.436+0000 7f197aa0d640 -1 bdev(0x5654e34ff800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T22:52:17.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:17.436+0000 7f197aa0d640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T22:52:20.589 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T22:52:20.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:20.706+0000 7fd2adbc5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T22:52:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:20.706+0000 7fd2adbc5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T22:52:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:20.706+0000 7fd2adbc5640 -1 bdev(0x55d2e056b800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T22:52:20.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:20.706+0000 7fd2adbc5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T22:52:23.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T22:52:23.896 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:23.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:23.984+0000 7fdfbb03e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-14T22:52:23.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:23.984+0000 7fdfbb03e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-14T22:52:23.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:23.984+0000 7fdfbb03e640 -1 bdev(0x561a686af800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-14T22:52:23.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:23.984+0000 7fdfbb03e640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-14T22:52:27.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-14T22:52:27.421 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:27.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:27.506+0000 7f91b71a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-14T22:52:27.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:27.506+0000 7f91b71a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-14T22:52:27.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:27.506+0000 7f91b71a4640 -1 bdev(0x557025af9800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-14T22:52:27.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-14T22:52:27.507+0000 7f91b71a4640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-14T22:52:30.667 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-14T22:52:30.697 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T22:52:30.697 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:30.697 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-14T22:52:30.727 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:30.727 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T22:52:30.797 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:30.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T22:52:30.867 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:30.867 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T22:52:30.937 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:30.937 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T22:52:31.007 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:31.007 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T22:52:31.038 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:31.039 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T22:52:31.107 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:31.108 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T22:52:31.175 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:31.175 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-14T22:52:31.242 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:31.242 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-14T22:52:31.308 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:31.308 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T22:52:31.327 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:31.327 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-14T22:52:31.365 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T22:52:31.365 DEBUG:teuthology.orchestra.run.smithi076:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T22:52:31.373 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T22:52:31.408 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.y] 2024-09-14T22:52:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCOE+ZmZ3jVJBAAGWwt96cRqAbI04QDbIktOA== 2024-09-14T22:52:31.408 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.0] 2024-09-14T22:52:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCRE+Zma3YENxAAi96DLh9zehpp+HPC3uaZCw== 2024-09-14T22:52:31.408 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.1] 2024-09-14T22:52:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCUE+ZmAwY6OBAAfdff5iQxCPD24wSRr9u0nw== 2024-09-14T22:52:31.408 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.2] 2024-09-14T22:52:31.408 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCXE+ZmT97kORAAy8jjwZf39jauQfPijXqLMw== 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.3] 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCbE+Zmg79wDhAAyYA+NaRv5hwJpzlH79Lc1A== 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.x] 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCOE+ZmH7ktKRAA3WXHc6NMFTUAOFsssO9IfA== 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.4] 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQChE+ZmNwIMGhAAc6TjC8AVPSMhmZAEinUUuQ== 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.5] 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCkE+ZmHqQ2KhAARfVadKuxCSjkhxc1YAffBg== 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.6] 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCnE+ZmmAS+OhAAsV1Zsh0kVwNNFpmUjI6yCA== 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.7] 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCrE+Zm4Z5AHhAAJOPr3VMouJp2qHCXaAF7qw== 2024-09-14T22:52:31.409 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-09-14T22:52:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCOE+ZmcijGKxAAz146nVx03nqWEq+3GK07uw== 2024-09-14T22:52:31.410 INFO:teuthology.orchestra.run.smithi076.stdout:[client.1] 2024-09-14T22:52:31.410 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCOE+ZmAEn/LhAAgQ98rQTQ7bDk+7LeOHO8Gw== 2024-09-14T22:52:31.428 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.y] 2024-09-14T22:52:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCOE+ZmZ3jVJBAAGWwt96cRqAbI04QDbIktOA== 2024-09-14T22:52:31.429 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2024-09-14T22:52:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCRE+Zma3YENxAAi96DLh9zehpp+HPC3uaZCw== 2024-09-14T22:52:31.429 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2024-09-14T22:52:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCUE+ZmAwY6OBAAfdff5iQxCPD24wSRr9u0nw== 2024-09-14T22:52:31.429 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2024-09-14T22:52:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCXE+ZmT97kORAAy8jjwZf39jauQfPijXqLMw== 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.3] 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCbE+Zmg79wDhAAyYA+NaRv5hwJpzlH79Lc1A== 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCOE+ZmH7ktKRAA3WXHc6NMFTUAOFsssO9IfA== 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.4] 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQChE+ZmNwIMGhAAc6TjC8AVPSMhmZAEinUUuQ== 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.5] 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCkE+ZmHqQ2KhAARfVadKuxCSjkhxc1YAffBg== 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.6] 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCnE+ZmmAS+OhAAsV1Zsh0kVwNNFpmUjI6yCA== 2024-09-14T22:52:31.430 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.7] 2024-09-14T22:52:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCrE+Zm4Z5AHhAAJOPr3VMouJp2qHCXaAF7qw== 2024-09-14T22:52:31.431 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-09-14T22:52:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCOE+ZmcijGKxAAz146nVx03nqWEq+3GK07uw== 2024-09-14T22:52:31.431 INFO:teuthology.orchestra.run.smithi114.stdout:[client.1] 2024-09-14T22:52:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQCOE+ZmAEn/LhAAgQ98rQTQ7bDk+7LeOHO8Gw== 2024-09-14T22:52:31.432 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T22:52:31.454 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T22:52:31.523 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.526 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.576 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.578 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.631 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.681 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.734 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T22:52:31.736 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T22:52:31.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.789 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.841 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.891 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.894 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.944 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.946 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T22:52:31.997 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T22:52:31.999 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T22:52:32.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T22:52:32.055 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T22:52:32.109 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T22:52:32.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T22:52:32.134 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T22:52:32.230 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T22:52:32.254 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T22:52:32.319 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T22:52:32.414 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T22:52:32.439 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T22:52:32.463 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T22:52:32.559 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T22:52:32.583 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:32.586 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T22:52:32.639 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T22:52:32.639 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T22:52:32.639 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T22:52:32.642 INFO:tasks.ceph.mon.a:Started 2024-09-14T22:52:32.642 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T22:52:32.642 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T22:52:32.644 INFO:tasks.ceph.mon.c:Started 2024-09-14T22:52:32.644 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T22:52:32.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T22:52:32.682 INFO:tasks.ceph.mon.b:Started 2024-09-14T22:52:32.682 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T22:52:32.682 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-14T22:52:32.682 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-14T22:52:32.685 INFO:tasks.ceph.mgr.y:Started 2024-09-14T22:52:32.685 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T22:52:32.685 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T22:52:32.687 INFO:tasks.ceph.mgr.x:Started 2024-09-14T22:52:32.687 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T22:52:32.687 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T22:52:38.032 INFO:teuthology.orchestra.run.smithi076.stderr:adjusted tunables profile to default 2024-09-14T22:52:38.044 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T22:52:38.044 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T22:52:38.044 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:38.044 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T22:52:38.070 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:38.071 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T22:52:38.135 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:38.135 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T22:52:38.199 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-14T22:52:38.199 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T22:52:38.264 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:38.264 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T22:52:38.294 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:38.294 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T22:52:38.363 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:38.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-14T22:52:38.431 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-14T22:52:38.431 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-14T22:52:38.497 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 45f61e20-37e6-4a13-b005-1f135f5e5a58 0 2024-09-14T22:52:38.720 INFO:teuthology.orchestra.run.smithi114.stdout:0 2024-09-14T22:52:38.729 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new bc72773d-098f-4ac5-89b5-13dae87b65b4 1 2024-09-14T22:52:38.861 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:38.860+0000 7f0c2332f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-14T22:52:38.871 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:38.869+0000 7facac931040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-14T22:52:38.901 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:38.900+0000 7f0c2332f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-14T22:52:38.911 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-14T22:52:38.912 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:38.911+0000 7facac931040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-14T22:52:38.923 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 2c2a50b0-e6ab-4593-9a26-badf1cc3dd34 2 2024-09-14T22:52:39.049 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:39.047+0000 7f0c2332f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-14T22:52:39.063 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:39.062+0000 7facac931040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-14T22:52:39.100 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:39.099+0000 7f0c2332f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-14T22:52:39.105 INFO:teuthology.orchestra.run.smithi114.stdout:2 2024-09-14T22:52:39.114 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:39.113+0000 7facac931040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-14T22:52:39.117 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 40d20d5f-5509-4a9f-8b13-c9e8da7339bc 3 2024-09-14T22:52:39.260 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:39.258+0000 7f0c2332f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-14T22:52:39.275 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:39.274+0000 7facac931040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-14T22:52:39.301 INFO:teuthology.orchestra.run.smithi114.stdout:3 2024-09-14T22:52:39.312 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 78bf5206-fbf7-4668-a27f-9eb867edc4e5 4 2024-09-14T22:52:39.375 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:39.374+0000 7f0c2332f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-14T22:52:39.385 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:39.384+0000 7facac931040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-14T22:52:39.495 INFO:teuthology.orchestra.run.smithi114.stdout:4 2024-09-14T22:52:39.507 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 72838af1-36f2-4cf6-a5ee-6daf5d4ea70a 5 2024-09-14T22:52:39.684 INFO:teuthology.orchestra.run.smithi114.stdout:5 2024-09-14T22:52:39.695 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 11be725a-e81f-4b8a-8a68-acb54e67b443 6 2024-09-14T22:52:39.702 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:39.700+0000 7facac931040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-14T22:52:39.704 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:39.703+0000 7f0c2332f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-14T22:52:39.751 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:39.750+0000 7facac931040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-14T22:52:39.763 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:39.761+0000 7f0c2332f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-14T22:52:39.879 INFO:teuthology.orchestra.run.smithi114.stdout:6 2024-09-14T22:52:39.891 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 94103011-716e-495f-aeb3-759071ce6f14 7 2024-09-14T22:52:39.903 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:39.901+0000 7facac931040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-14T22:52:39.919 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:39.918+0000 7f0c2332f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-14T22:52:40.097 INFO:teuthology.orchestra.run.smithi114.stdout:7 2024-09-14T22:52:40.109 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T22:52:40.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T22:52:40.112 INFO:tasks.ceph.osd.0:Started 2024-09-14T22:52:40.112 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T22:52:40.112 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T22:52:40.114 INFO:tasks.ceph.osd.1:Started 2024-09-14T22:52:40.114 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T22:52:40.115 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T22:52:40.117 INFO:tasks.ceph.osd.2:Started 2024-09-14T22:52:40.117 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T22:52:40.117 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T22:52:40.119 INFO:tasks.ceph.osd.3:Started 2024-09-14T22:52:40.119 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T22:52:40.120 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T22:52:40.122 INFO:tasks.ceph.osd.4:Started 2024-09-14T22:52:40.122 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T22:52:40.122 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T22:52:40.124 INFO:tasks.ceph.osd.5:Started 2024-09-14T22:52:40.124 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-14T22:52:40.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-14T22:52:40.126 INFO:tasks.ceph.osd.6:Started 2024-09-14T22:52:40.127 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-14T22:52:40.127 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-14T22:52:40.129 INFO:tasks.ceph.osd.7:Started 2024-09-14T22:52:40.129 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:52:40.324 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:52:40.324 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":10,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:52:40.095883+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T22:52:40.095883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:52:40.335 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T22:52:40.335 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T22:52:40.880 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:40.878+0000 7f0c2332f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-14T22:52:40.911 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:40.910+0000 7facac931040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-14T22:52:40.931 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:40.930+0000 7f0c2332f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-14T22:52:40.965 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:40.964+0000 7facac931040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-14T22:52:41.072 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:52:41.070+0000 7fcd5042a640 -1 Falling back to public interface 2024-09-14T22:52:41.141 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T22:52:41.298 INFO:teuthology.misc.health.smithi076.stdout: 2024-09-14T22:52:41.298 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":10,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:52:40.095883+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T22:52:40.095883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:52:41.301 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.299+0000 7f0c2332f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-14T22:52:41.308 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T22:52:41.346 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.345+0000 7facac931040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-14T22:52:41.370 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:52:41.368+0000 7faddf42c640 -1 Falling back to public interface 2024-09-14T22:52:41.387 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.386+0000 7f0c2332f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-14T22:52:41.431 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.429+0000 7facac931040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-14T22:52:41.494 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.493+0000 7f0c2332f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-14T22:52:41.536 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.535+0000 7facac931040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-14T22:52:41.541 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.539+0000 7f0c2332f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-14T22:52:41.584 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.582+0000 7facac931040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-14T22:52:41.584 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:52:41.583+0000 7f747dfb0640 -1 Falling back to public interface 2024-09-14T22:52:41.617 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:52:41.615+0000 7f2f45c24640 -1 Falling back to public interface 2024-09-14T22:52:41.659 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.657+0000 7f0c2332f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-14T22:52:41.697 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.696+0000 7facac931040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-14T22:52:41.711 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.709+0000 7f0c2332f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-14T22:52:41.750 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.749+0000 7facac931040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-14T22:52:41.755 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.753+0000 7f0c2332f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-14T22:52:41.795 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.793+0000 7facac931040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-14T22:52:41.816 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:52:41.814+0000 7f25201bb640 -1 Falling back to public interface 2024-09-14T22:52:41.824 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:52:41.821+0000 7f236de0d640 -1 Falling back to public interface 2024-09-14T22:52:41.866 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.865+0000 7f0c2332f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-14T22:52:41.868 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:52:41.866+0000 7f3888c0c640 -1 Falling back to public interface 2024-09-14T22:52:41.910 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.909+0000 7facac931040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-14T22:52:41.917 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.916+0000 7f0c2332f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-14T22:52:41.961 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:41.960+0000 7facac931040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-14T22:52:41.969 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:41.967+0000 7f0c2332f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-14T22:52:42.012 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:42.011+0000 7facac931040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-14T22:52:42.034 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:42.033+0000 7f0c2332f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-14T22:52:42.078 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:42.076+0000 7f0c2332f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-14T22:52:42.079 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:42.078+0000 7facac931040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-14T22:52:42.103 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:52:42.102+0000 7f12233cc640 -1 Falling back to public interface 2024-09-14T22:52:42.125 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:42.123+0000 7f0c2332f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-14T22:52:42.125 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:42.124+0000 7facac931040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-14T22:52:42.170 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:42.169+0000 7facac931040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-14T22:52:42.775 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:42.773+0000 7f0c2332f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-14T22:52:42.830 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-14T22:52:42.828+0000 7facac931040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-14T22:52:43.789 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:43.787+0000 7f0bc32f0640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T22:52:43.789 INFO:tasks.ceph.mgr.x.smithi114.stderr:2024-09-14T22:52:43.788+0000 7f0bc32f0640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T22:52:45.148 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:52:45.147+0000 7fcd5042a640 -1 osd.0 0 log_to_monitors true 2024-09-14T22:52:45.389 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:52:45.388+0000 7f747dfb0640 -1 osd.1 0 log_to_monitors true 2024-09-14T22:52:45.432 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:52:45.431+0000 7f2f45c24640 -1 osd.7 0 log_to_monitors true 2024-09-14T22:52:45.445 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:52:45.443+0000 7faddf42c640 -1 osd.5 0 log_to_monitors true 2024-09-14T22:52:45.651 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:52:45.650+0000 7f3888c0c640 -1 osd.6 0 log_to_monitors true 2024-09-14T22:52:45.845 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:52:45.844+0000 7f236de0d640 -1 osd.2 0 log_to_monitors true 2024-09-14T22:52:45.899 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:52:45.898+0000 7f25201bb640 -1 osd.3 0 log_to_monitors true 2024-09-14T22:52:46.123 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:52:46.121+0000 7f12233cc640 -1 osd.4 0 log_to_monitors true 2024-09-14T22:52:48.113 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T22:52:48.407 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:52:48.406+0000 7f7479f3f640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T22:52:48.504 INFO:teuthology.misc.health.smithi076.stdout: 2024-09-14T22:52:48.505 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":13,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:52:47.788375+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T22:52:40.095883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:52:48.521 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T22:52:48.523 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:52:48.522+0000 7f7471d5f640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:52:48.530 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:52:48.529+0000 7fcd4cbcd640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T22:52:48.552 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:52:48.551+0000 7fcd441d9640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:52:48.664 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:52:48.663+0000 7faddaa3a640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T22:52:48.677 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:52:48.676+0000 7fadd285a640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:52:48.746 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:52:48.744+0000 7f2f41bb3640 -1 osd.7 0 waiting for initial osdmap 2024-09-14T22:52:48.761 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:52:48.760+0000 7f2f399d3640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:52:48.786 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:52:48.785+0000 7f38845d9640 -1 osd.6 0 waiting for initial osdmap 2024-09-14T22:52:48.804 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:52:48.802+0000 7f387c3f9640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:52:48.986 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:52:48.984+0000 7f121f35b640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T22:52:48.996 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:52:48.995+0000 7f121717b640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:52:49.154 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:52:49.152+0000 7f23696fe640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T22:52:49.164 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:52:49.163+0000 7f236151e640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:52:49.215 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:52:49.214+0000 7f251c14a640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T22:52:49.230 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:52:49.229+0000 7f2513f6a640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:52:55.328 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T22:52:55.608 INFO:teuthology.misc.health.smithi076.stdout: 2024-09-14T22:52:55.608 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":18,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:52:53.501826+0000","last_up_change":"2024-09-14T22:52:49.483913+0000","last_in_change":"2024-09-14T22:52:40.095883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:52:50.813186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6801","nonce":2488220599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6803","nonce":2488220599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6807","nonce":2488220599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6805","nonce":2488220599}]},"public_addr":"172.21.15.76:6801/2488220599","cluster_addr":"172.21.15.76:6803/2488220599","heartbeat_back_addr":"172.21.15.76:6807/2488220599","heartbeat_front_addr":"172.21.15.76:6805/2488220599","state":["exists","up"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6809","nonce":522829438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6811","nonce":522829438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6815","nonce":522829438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6813","nonce":522829438}]},"public_addr":"172.21.15.76:6809/522829438","cluster_addr":"172.21.15.76:6811/522829438","heartbeat_back_addr":"172.21.15.76:6815/522829438","heartbeat_front_addr":"172.21.15.76:6813/522829438","state":["exists","up"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6825","nonce":3214755929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6827","nonce":3214755929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6831","nonce":3214755929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6829","nonce":3214755929}]},"public_addr":"172.21.15.76:6825/3214755929","cluster_addr":"172.21.15.76:6827/3214755929","heartbeat_back_addr":"172.21.15.76:6831/3214755929","heartbeat_front_addr":"172.21.15.76:6829/3214755929","state":["exists","up"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6817","nonce":2699236014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6819","nonce":2699236014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6823","nonce":2699236014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6821","nonce":2699236014}]},"public_addr":"172.21.15.76:6817/2699236014","cluster_addr":"172.21.15.76:6819/2699236014","heartbeat_back_addr":"172.21.15.76:6823/2699236014","heartbeat_front_addr":"172.21.15.76:6821/2699236014","state":["exists","up"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6825","nonce":2097675638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6827","nonce":2097675638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6831","nonce":2097675638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6829","nonce":2097675638}]},"public_addr":"172.21.15.114:6825/2097675638","cluster_addr":"172.21.15.114:6827/2097675638","heartbeat_back_addr":"172.21.15.114:6831/2097675638","heartbeat_front_addr":"172.21.15.114:6829/2097675638","state":["exists","up"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6801","nonce":3469663218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6803","nonce":3469663218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6807","nonce":3469663218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6805","nonce":3469663218}]},"public_addr":"172.21.15.114:6801/3469663218","cluster_addr":"172.21.15.114:6803/3469663218","heartbeat_back_addr":"172.21.15.114:6807/3469663218","heartbeat_front_addr":"172.21.15.114:6805/3469663218","state":["exists","up"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6817","nonce":2551471767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6819","nonce":2551471767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6823","nonce":2551471767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6821","nonce":2551471767}]},"public_addr":"172.21.15.114:6817/2551471767","cluster_addr":"172.21.15.114:6819/2551471767","heartbeat_back_addr":"172.21.15.114:6823/2551471767","heartbeat_front_addr":"172.21.15.114:6821/2551471767","state":["exists","up"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6809","nonce":941014947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6811","nonce":941014947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6815","nonce":941014947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6813","nonce":941014947}]},"public_addr":"172.21.15.114:6809/941014947","cluster_addr":"172.21.15.114:6811/941014947","heartbeat_back_addr":"172.21.15.114:6815/941014947","heartbeat_front_addr":"172.21.15.114:6813/941014947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.133153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.415439+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.880448+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.879405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:47.079602+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.493580+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.693553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.442044+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:52:55.620 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-14T22:52:55.620 INFO:tasks.ceph:Creating RBD pool 2024-09-14T22:52:55.620 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T22:52:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'rbd' created 2024-09-14T22:52:56.532 DEBUG:teuthology.orchestra.run.smithi076:> rbd --cluster ceph pool init rbd 2024-09-14T22:52:59.544 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T22:52:59.544 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T22:52:59.544 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T22:52:59.544 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T22:52:59.545 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T22:52:59.877 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:52:59.887 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":3014932084},{"type":"v1","addr":"172.21.15.114:6833","nonce":3014932084}]},"active_addr":"172.21.15.114:6833/3014932084","active_change":"2024-09-14T22:52:42.776275+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":3100072810}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":3378365495}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":1049733357}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":2261938578}]}]} 2024-09-14T22:52:59.891 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T22:52:59.891 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T22:52:59.891 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:53:00.169 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:00.169 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:52:59.526193+0000","last_up_change":"2024-09-14T22:52:49.483913+0000","last_in_change":"2024-09-14T22:52:40.095883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:52:50.813186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:52:55.912547+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6801","nonce":2488220599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6803","nonce":2488220599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6807","nonce":2488220599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6805","nonce":2488220599}]},"public_addr":"172.21.15.76:6801/2488220599","cluster_addr":"172.21.15.76:6803/2488220599","heartbeat_back_addr":"172.21.15.76:6807/2488220599","heartbeat_front_addr":"172.21.15.76:6805/2488220599","state":["exists","up"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6809","nonce":522829438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6811","nonce":522829438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6815","nonce":522829438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6813","nonce":522829438}]},"public_addr":"172.21.15.76:6809/522829438","cluster_addr":"172.21.15.76:6811/522829438","heartbeat_back_addr":"172.21.15.76:6815/522829438","heartbeat_front_addr":"172.21.15.76:6813/522829438","state":["exists","up"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6825","nonce":3214755929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6827","nonce":3214755929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6831","nonce":3214755929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6829","nonce":3214755929}]},"public_addr":"172.21.15.76:6825/3214755929","cluster_addr":"172.21.15.76:6827/3214755929","heartbeat_back_addr":"172.21.15.76:6831/3214755929","heartbeat_front_addr":"172.21.15.76:6829/3214755929","state":["exists","up"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6817","nonce":2699236014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6819","nonce":2699236014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6823","nonce":2699236014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6821","nonce":2699236014}]},"public_addr":"172.21.15.76:6817/2699236014","cluster_addr":"172.21.15.76:6819/2699236014","heartbeat_back_addr":"172.21.15.76:6823/2699236014","heartbeat_front_addr":"172.21.15.76:6821/2699236014","state":["exists","up"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6825","nonce":2097675638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6827","nonce":2097675638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6831","nonce":2097675638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6829","nonce":2097675638}]},"public_addr":"172.21.15.114:6825/2097675638","cluster_addr":"172.21.15.114:6827/2097675638","heartbeat_back_addr":"172.21.15.114:6831/2097675638","heartbeat_front_addr":"172.21.15.114:6829/2097675638","state":["exists","up"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6801","nonce":3469663218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6803","nonce":3469663218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6807","nonce":3469663218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6805","nonce":3469663218}]},"public_addr":"172.21.15.114:6801/3469663218","cluster_addr":"172.21.15.114:6803/3469663218","heartbeat_back_addr":"172.21.15.114:6807/3469663218","heartbeat_front_addr":"172.21.15.114:6805/3469663218","state":["exists","up"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6817","nonce":2551471767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6819","nonce":2551471767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6823","nonce":2551471767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6821","nonce":2551471767}]},"public_addr":"172.21.15.114:6817/2551471767","cluster_addr":"172.21.15.114:6819/2551471767","heartbeat_back_addr":"172.21.15.114:6823/2551471767","heartbeat_front_addr":"172.21.15.114:6821/2551471767","state":["exists","up"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6809","nonce":941014947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6811","nonce":941014947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6815","nonce":941014947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6813","nonce":941014947}]},"public_addr":"172.21.15.114:6809/941014947","cluster_addr":"172.21.15.114:6811/941014947","heartbeat_back_addr":"172.21.15.114:6815/941014947","heartbeat_front_addr":"172.21.15.114:6813/941014947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.133153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.415439+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.880448+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.879405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:47.079602+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.493580+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.693553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.442044+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:53:00.181 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T22:53:00.182 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:53:00.457 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:00.457 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:52:59.526193+0000","last_up_change":"2024-09-14T22:52:49.483913+0000","last_in_change":"2024-09-14T22:52:40.095883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:52:50.813186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:52:55.912547+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6801","nonce":2488220599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6803","nonce":2488220599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6807","nonce":2488220599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6805","nonce":2488220599}]},"public_addr":"172.21.15.76:6801/2488220599","cluster_addr":"172.21.15.76:6803/2488220599","heartbeat_back_addr":"172.21.15.76:6807/2488220599","heartbeat_front_addr":"172.21.15.76:6805/2488220599","state":["exists","up"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6809","nonce":522829438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6811","nonce":522829438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6815","nonce":522829438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6813","nonce":522829438}]},"public_addr":"172.21.15.76:6809/522829438","cluster_addr":"172.21.15.76:6811/522829438","heartbeat_back_addr":"172.21.15.76:6815/522829438","heartbeat_front_addr":"172.21.15.76:6813/522829438","state":["exists","up"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6825","nonce":3214755929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6827","nonce":3214755929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6831","nonce":3214755929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6829","nonce":3214755929}]},"public_addr":"172.21.15.76:6825/3214755929","cluster_addr":"172.21.15.76:6827/3214755929","heartbeat_back_addr":"172.21.15.76:6831/3214755929","heartbeat_front_addr":"172.21.15.76:6829/3214755929","state":["exists","up"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6817","nonce":2699236014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6819","nonce":2699236014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6823","nonce":2699236014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6821","nonce":2699236014}]},"public_addr":"172.21.15.76:6817/2699236014","cluster_addr":"172.21.15.76:6819/2699236014","heartbeat_back_addr":"172.21.15.76:6823/2699236014","heartbeat_front_addr":"172.21.15.76:6821/2699236014","state":["exists","up"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6825","nonce":2097675638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6827","nonce":2097675638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6831","nonce":2097675638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":2097675638},{"type":"v1","addr":"172.21.15.114:6829","nonce":2097675638}]},"public_addr":"172.21.15.114:6825/2097675638","cluster_addr":"172.21.15.114:6827/2097675638","heartbeat_back_addr":"172.21.15.114:6831/2097675638","heartbeat_front_addr":"172.21.15.114:6829/2097675638","state":["exists","up"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6801","nonce":3469663218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6803","nonce":3469663218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6807","nonce":3469663218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6805","nonce":3469663218}]},"public_addr":"172.21.15.114:6801/3469663218","cluster_addr":"172.21.15.114:6803/3469663218","heartbeat_back_addr":"172.21.15.114:6807/3469663218","heartbeat_front_addr":"172.21.15.114:6805/3469663218","state":["exists","up"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6817","nonce":2551471767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6819","nonce":2551471767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6823","nonce":2551471767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6821","nonce":2551471767}]},"public_addr":"172.21.15.114:6817/2551471767","cluster_addr":"172.21.15.114:6819/2551471767","heartbeat_back_addr":"172.21.15.114:6823/2551471767","heartbeat_front_addr":"172.21.15.114:6821/2551471767","state":["exists","up"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6809","nonce":941014947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6811","nonce":941014947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6815","nonce":941014947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6813","nonce":941014947}]},"public_addr":"172.21.15.114:6809/941014947","cluster_addr":"172.21.15.114:6811/941014947","heartbeat_back_addr":"172.21.15.114:6815/941014947","heartbeat_front_addr":"172.21.15.114:6813/941014947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.133153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.415439+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.880448+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.879405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:47.079602+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.493580+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.693553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.442044+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:53:00.470 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T22:53:00.471 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T22:53:00.471 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T22:53:00.471 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T22:53:00.472 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T22:53:00.472 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T22:53:00.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T22:53:00.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T22:53:00.708 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:00.708 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T22:53:00.713 INFO:teuthology.orchestra.run.smithi076.stdout:60129542151 2024-09-14T22:53:00.714 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:00.714 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T22:53:00.715 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T22:53:00.715 INFO:teuthology.orchestra.run.smithi076.stdout:60129542151 2024-09-14T22:53:00.716 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T22:53:00.716 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:00.717 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:00.717 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T22:53:00.718 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T22:53:00.718 INFO:teuthology.orchestra.run.smithi076.stdout:60129542151 2024-09-14T22:53:00.719 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T22:53:00.720 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:00.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T22:53:01.294 INFO:teuthology.orchestra.run.smithi076.stdout:60129542148 2024-09-14T22:53:01.301 INFO:teuthology.orchestra.run.smithi076.stdout:60129542148 2024-09-14T22:53:01.312 INFO:teuthology.orchestra.run.smithi076.stdout:60129542148 2024-09-14T22:53:01.313 INFO:teuthology.orchestra.run.smithi076.stdout:60129542148 2024-09-14T22:53:01.315 INFO:teuthology.orchestra.run.smithi076.stdout:60129542149 2024-09-14T22:53:01.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.2 2024-09-14T22:53:01.320 INFO:teuthology.orchestra.run.smithi076.stdout:60129542149 2024-09-14T22:53:01.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.5 2024-09-14T22:53:01.325 INFO:teuthology.orchestra.run.smithi076.stdout:60129542148 2024-09-14T22:53:01.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.1 2024-09-14T22:53:01.336 INFO:teuthology.orchestra.run.smithi076.stdout:60129542149 2024-09-14T22:53:01.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.7 2024-09-14T22:53:01.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.3 2024-09-14T22:53:01.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.6 2024-09-14T22:53:01.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.4 2024-09-14T22:53:01.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.0 2024-09-14T22:53:02.317 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T22:53:02.322 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T22:53:02.334 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T22:53:02.338 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T22:53:02.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T22:53:02.342 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T22:53:02.343 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T22:53:02.350 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T22:53:02.897 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:02.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2024-09-14T22:53:02.928 DEBUG:teuthology.parallel:result is None 2024-09-14T22:53:02.938 INFO:teuthology.orchestra.run.smithi076.stdout:60129542151 2024-09-14T22:53:02.950 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.6 2024-09-14T22:53:02.951 DEBUG:teuthology.parallel:result is None 2024-09-14T22:53:02.964 INFO:teuthology.orchestra.run.smithi076.stdout:60129542151 2024-09-14T22:53:02.977 INFO:teuthology.orchestra.run.smithi076.stdout:60129542151 2024-09-14T22:53:02.977 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:02.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-09-14T22:53:02.983 DEBUG:teuthology.parallel:result is None 2024-09-14T22:53:02.984 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:02.987 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:02.988 INFO:teuthology.orchestra.run.smithi076.stdout:60129542150 2024-09-14T22:53:02.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2024-09-14T22:53:02.998 DEBUG:teuthology.parallel:result is None 2024-09-14T22:53:02.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-09-14T22:53:02.999 DEBUG:teuthology.parallel:result is None 2024-09-14T22:53:03.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-09-14T22:53:03.004 DEBUG:teuthology.parallel:result is None 2024-09-14T22:53:03.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-09-14T22:53:03.005 DEBUG:teuthology.parallel:result is None 2024-09-14T22:53:03.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2024-09-14T22:53:03.008 DEBUG:teuthology.parallel:result is None 2024-09-14T22:53:03.008 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T22:53:03.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:53:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:03.321 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:53:03.332 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T22:53:02.784196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219128,"kb_used_data":2880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755592,"statfs":{"total":773094113280,"available":772869726208,"internally_reserved":0,"allocated":2949120,"data_stored":1771222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.262189"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.822969+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:52:59.822969+0000","last_peered":"2024-09-14T22:52:59.822969+0000","last_clean":"2024-09-14T22:52:59.822969+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:52:59.822969+0000","last_undegraded":"2024-09-14T22:52:59.822969+0000","last_fullsized":"2024-09-14T22:52:59.822969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.538561+0000","last_change":"2024-09-14T22:52:59.538561+0000","last_active":"2024-09-14T22:52:59.538561+0000","last_peered":"2024-09-14T22:52:59.538561+0000","last_clean":"2024-09-14T22:52:59.538561+0000","last_became_active":"2024-09-14T22:52:57.530436+0000","last_became_peered":"2024-09-14T22:52:57.530436+0000","last_unstale":"2024-09-14T22:52:59.538561+0000","last_undegraded":"2024-09-14T22:52:59.538561+0000","last_fullsized":"2024-09-14T22:52:59.538561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:29.562282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415386+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:00.415386+0000","last_peered":"2024-09-14T22:53:00.415386+0000","last_clean":"2024-09-14T22:53:00.415386+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:00.415386+0000","last_undegraded":"2024-09-14T22:53:00.415386+0000","last_fullsized":"2024-09-14T22:53:00.415386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.538529+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:52:59.538529+0000","last_peered":"2024-09-14T22:52:59.538529+0000","last_clean":"2024-09-14T22:52:59.538529+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:52:59.538529+0000","last_undegraded":"2024-09-14T22:52:59.538529+0000","last_fullsized":"2024-09-14T22:52:59.538529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.537632+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:52:59.537632+0000","last_peered":"2024-09-14T22:52:59.537632+0000","last_clean":"2024-09-14T22:52:59.537632+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:52:59.537632+0000","last_undegraded":"2024-09-14T22:52:59.537632+0000","last_fullsized":"2024-09-14T22:52:59.537632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.039690+0000","last_change":"2024-09-14T22:53:00.039871+0000","last_active":"2024-09-14T22:53:00.039690+0000","last_peered":"2024-09-14T22:53:00.039690+0000","last_clean":"2024-09-14T22:53:00.039690+0000","last_became_active":"2024-09-14T22:52:57.694658+0000","last_became_peered":"2024-09-14T22:52:57.694658+0000","last_unstale":"2024-09-14T22:53:00.039690+0000","last_undegraded":"2024-09-14T22:53:00.039690+0000","last_fullsized":"2024-09-14T22:53:00.039690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:09.229943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415434+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:00.415434+0000","last_peered":"2024-09-14T22:53:00.415434+0000","last_clean":"2024-09-14T22:53:00.415434+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:00.415434+0000","last_undegraded":"2024-09-14T22:53:00.415434+0000","last_fullsized":"2024-09-14T22:53:00.415434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.536124+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:52:59.536124+0000","last_peered":"2024-09-14T22:52:59.536124+0000","last_clean":"2024-09-14T22:52:59.536124+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:52:59.536124+0000","last_undegraded":"2024-09-14T22:52:59.536124+0000","last_fullsized":"2024-09-14T22:52:59.536124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415143+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:00.415143+0000","last_peered":"2024-09-14T22:53:00.415143+0000","last_clean":"2024-09-14T22:53:00.415143+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:00.415143+0000","last_undegraded":"2024-09-14T22:53:00.415143+0000","last_fullsized":"2024-09-14T22:53:00.415143+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":661258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":667090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:53:03.333 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:53:03.608 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:03.608 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:53:03.620 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T22:53:02.784196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219128,"kb_used_data":2880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755592,"statfs":{"total":773094113280,"available":772869726208,"internally_reserved":0,"allocated":2949120,"data_stored":1771222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.262189"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.822969+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:52:59.822969+0000","last_peered":"2024-09-14T22:52:59.822969+0000","last_clean":"2024-09-14T22:52:59.822969+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:52:59.822969+0000","last_undegraded":"2024-09-14T22:52:59.822969+0000","last_fullsized":"2024-09-14T22:52:59.822969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.538561+0000","last_change":"2024-09-14T22:52:59.538561+0000","last_active":"2024-09-14T22:52:59.538561+0000","last_peered":"2024-09-14T22:52:59.538561+0000","last_clean":"2024-09-14T22:52:59.538561+0000","last_became_active":"2024-09-14T22:52:57.530436+0000","last_became_peered":"2024-09-14T22:52:57.530436+0000","last_unstale":"2024-09-14T22:52:59.538561+0000","last_undegraded":"2024-09-14T22:52:59.538561+0000","last_fullsized":"2024-09-14T22:52:59.538561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:29.562282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415386+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:00.415386+0000","last_peered":"2024-09-14T22:53:00.415386+0000","last_clean":"2024-09-14T22:53:00.415386+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:00.415386+0000","last_undegraded":"2024-09-14T22:53:00.415386+0000","last_fullsized":"2024-09-14T22:53:00.415386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.538529+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:52:59.538529+0000","last_peered":"2024-09-14T22:52:59.538529+0000","last_clean":"2024-09-14T22:52:59.538529+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:52:59.538529+0000","last_undegraded":"2024-09-14T22:52:59.538529+0000","last_fullsized":"2024-09-14T22:52:59.538529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.537632+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:52:59.537632+0000","last_peered":"2024-09-14T22:52:59.537632+0000","last_clean":"2024-09-14T22:52:59.537632+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:52:59.537632+0000","last_undegraded":"2024-09-14T22:52:59.537632+0000","last_fullsized":"2024-09-14T22:52:59.537632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.039690+0000","last_change":"2024-09-14T22:53:00.039871+0000","last_active":"2024-09-14T22:53:00.039690+0000","last_peered":"2024-09-14T22:53:00.039690+0000","last_clean":"2024-09-14T22:53:00.039690+0000","last_became_active":"2024-09-14T22:52:57.694658+0000","last_became_peered":"2024-09-14T22:52:57.694658+0000","last_unstale":"2024-09-14T22:53:00.039690+0000","last_undegraded":"2024-09-14T22:53:00.039690+0000","last_fullsized":"2024-09-14T22:53:00.039690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:09.229943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415434+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:00.415434+0000","last_peered":"2024-09-14T22:53:00.415434+0000","last_clean":"2024-09-14T22:53:00.415434+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:00.415434+0000","last_undegraded":"2024-09-14T22:53:00.415434+0000","last_fullsized":"2024-09-14T22:53:00.415434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.536124+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:52:59.536124+0000","last_peered":"2024-09-14T22:52:59.536124+0000","last_clean":"2024-09-14T22:52:59.536124+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:52:59.536124+0000","last_undegraded":"2024-09-14T22:52:59.536124+0000","last_fullsized":"2024-09-14T22:52:59.536124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415143+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:00.415143+0000","last_peered":"2024-09-14T22:53:00.415143+0000","last_clean":"2024-09-14T22:53:00.415143+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:00.415143+0000","last_undegraded":"2024-09-14T22:53:00.415143+0000","last_fullsized":"2024-09-14T22:53:00.415143+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":661258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":667090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:53:03.622 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T22:53:03.622 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T22:53:03.622 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T22:53:03.622 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T22:53:03.932 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:03.933 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T22:53:03.945 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T22:53:03.945 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T22:53:03.961 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T22:53:03.961 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ecab7be58963a5bd03910cd73eef66405d7fdd50'}} 2024-09-14T22:53:03.962 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T22:53:03.962 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T22:53:04.256 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T22:53:04.256 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T22:53:04.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:53:04.532 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:04.533 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:53:04.544 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-14T22:53:02.784196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219128,"kb_used_data":2880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755592,"statfs":{"total":773094113280,"available":772869726208,"internally_reserved":0,"allocated":2949120,"data_stored":1771222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.262189"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.822969+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:52:59.822969+0000","last_peered":"2024-09-14T22:52:59.822969+0000","last_clean":"2024-09-14T22:52:59.822969+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:52:59.822969+0000","last_undegraded":"2024-09-14T22:52:59.822969+0000","last_fullsized":"2024-09-14T22:52:59.822969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.538561+0000","last_change":"2024-09-14T22:52:59.538561+0000","last_active":"2024-09-14T22:52:59.538561+0000","last_peered":"2024-09-14T22:52:59.538561+0000","last_clean":"2024-09-14T22:52:59.538561+0000","last_became_active":"2024-09-14T22:52:57.530436+0000","last_became_peered":"2024-09-14T22:52:57.530436+0000","last_unstale":"2024-09-14T22:52:59.538561+0000","last_undegraded":"2024-09-14T22:52:59.538561+0000","last_fullsized":"2024-09-14T22:52:59.538561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:29.562282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415386+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:00.415386+0000","last_peered":"2024-09-14T22:53:00.415386+0000","last_clean":"2024-09-14T22:53:00.415386+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:00.415386+0000","last_undegraded":"2024-09-14T22:53:00.415386+0000","last_fullsized":"2024-09-14T22:53:00.415386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.538529+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:52:59.538529+0000","last_peered":"2024-09-14T22:52:59.538529+0000","last_clean":"2024-09-14T22:52:59.538529+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:52:59.538529+0000","last_undegraded":"2024-09-14T22:52:59.538529+0000","last_fullsized":"2024-09-14T22:52:59.538529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.537632+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:52:59.537632+0000","last_peered":"2024-09-14T22:52:59.537632+0000","last_clean":"2024-09-14T22:52:59.537632+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:52:59.537632+0000","last_undegraded":"2024-09-14T22:52:59.537632+0000","last_fullsized":"2024-09-14T22:52:59.537632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.039690+0000","last_change":"2024-09-14T22:53:00.039871+0000","last_active":"2024-09-14T22:53:00.039690+0000","last_peered":"2024-09-14T22:53:00.039690+0000","last_clean":"2024-09-14T22:53:00.039690+0000","last_became_active":"2024-09-14T22:52:57.694658+0000","last_became_peered":"2024-09-14T22:52:57.694658+0000","last_unstale":"2024-09-14T22:53:00.039690+0000","last_undegraded":"2024-09-14T22:53:00.039690+0000","last_fullsized":"2024-09-14T22:53:00.039690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:09.229943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415434+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:00.415434+0000","last_peered":"2024-09-14T22:53:00.415434+0000","last_clean":"2024-09-14T22:53:00.415434+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:00.415434+0000","last_undegraded":"2024-09-14T22:53:00.415434+0000","last_fullsized":"2024-09-14T22:53:00.415434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.536124+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:52:59.536124+0000","last_peered":"2024-09-14T22:52:59.536124+0000","last_clean":"2024-09-14T22:52:59.536124+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:52:59.536124+0000","last_undegraded":"2024-09-14T22:52:59.536124+0000","last_fullsized":"2024-09-14T22:52:59.536124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415143+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:00.415143+0000","last_peered":"2024-09-14T22:53:00.415143+0000","last_clean":"2024-09-14T22:53:00.415143+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:00.415143+0000","last_undegraded":"2024-09-14T22:53:00.415143+0000","last_fullsized":"2024-09-14T22:53:00.415143+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":661258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":667090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:53:04.545 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:53:04.820 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:04.821 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:53:04.832 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T22:53:04.784710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219128,"kb_used_data":2880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755592,"statfs":{"total":773094113280,"available":772869726208,"internally_reserved":0,"allocated":2949120,"data_stored":1771222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.258159"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.822969+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:52:59.822969+0000","last_peered":"2024-09-14T22:52:59.822969+0000","last_clean":"2024-09-14T22:52:59.822969+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:52:59.822969+0000","last_undegraded":"2024-09-14T22:52:59.822969+0000","last_fullsized":"2024-09-14T22:52:59.822969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.538561+0000","last_change":"2024-09-14T22:52:59.538561+0000","last_active":"2024-09-14T22:52:59.538561+0000","last_peered":"2024-09-14T22:52:59.538561+0000","last_clean":"2024-09-14T22:52:59.538561+0000","last_became_active":"2024-09-14T22:52:57.530436+0000","last_became_peered":"2024-09-14T22:52:57.530436+0000","last_unstale":"2024-09-14T22:52:59.538561+0000","last_undegraded":"2024-09-14T22:52:59.538561+0000","last_fullsized":"2024-09-14T22:52:59.538561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:29.562282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415386+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:00.415386+0000","last_peered":"2024-09-14T22:53:00.415386+0000","last_clean":"2024-09-14T22:53:00.415386+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:00.415386+0000","last_undegraded":"2024-09-14T22:53:00.415386+0000","last_fullsized":"2024-09-14T22:53:00.415386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.538529+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:52:59.538529+0000","last_peered":"2024-09-14T22:52:59.538529+0000","last_clean":"2024-09-14T22:52:59.538529+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:52:59.538529+0000","last_undegraded":"2024-09-14T22:52:59.538529+0000","last_fullsized":"2024-09-14T22:52:59.538529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.537632+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:52:59.537632+0000","last_peered":"2024-09-14T22:52:59.537632+0000","last_clean":"2024-09-14T22:52:59.537632+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:52:59.537632+0000","last_undegraded":"2024-09-14T22:52:59.537632+0000","last_fullsized":"2024-09-14T22:52:59.537632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.039690+0000","last_change":"2024-09-14T22:53:00.039871+0000","last_active":"2024-09-14T22:53:00.039690+0000","last_peered":"2024-09-14T22:53:00.039690+0000","last_clean":"2024-09-14T22:53:00.039690+0000","last_became_active":"2024-09-14T22:52:57.694658+0000","last_became_peered":"2024-09-14T22:52:57.694658+0000","last_unstale":"2024-09-14T22:53:00.039690+0000","last_undegraded":"2024-09-14T22:53:00.039690+0000","last_fullsized":"2024-09-14T22:53:00.039690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:09.229943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415434+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:00.415434+0000","last_peered":"2024-09-14T22:53:00.415434+0000","last_clean":"2024-09-14T22:53:00.415434+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:00.415434+0000","last_undegraded":"2024-09-14T22:53:00.415434+0000","last_fullsized":"2024-09-14T22:53:00.415434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:52:59.536124+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:52:59.536124+0000","last_peered":"2024-09-14T22:52:59.536124+0000","last_clean":"2024-09-14T22:52:59.536124+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:52:59.536124+0000","last_undegraded":"2024-09-14T22:52:59.536124+0000","last_fullsized":"2024-09-14T22:52:59.536124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T22:53:00.415143+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:00.415143+0000","last_peered":"2024-09-14T22:53:00.415143+0000","last_clean":"2024-09-14T22:53:00.415143+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:00.415143+0000","last_undegraded":"2024-09-14T22:53:00.415143+0000","last_fullsized":"2024-09-14T22:53:00.415143+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":661258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":819200,"data_stored":667090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:53:04.833 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T22:53:04.833 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T22:53:05.110 INFO:teuthology.orchestra.run.smithi076.stdout:epoch 23 2024-09-14T22:53:05.110 INFO:teuthology.orchestra.run.smithi076.stdout:fsid 9a42a40e-8d25-4801-9835-f823a294bdbe 2024-09-14T22:53:05.110 INFO:teuthology.orchestra.run.smithi076.stdout:created 2024-09-14T22:52:37.944451+0000 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:modified 2024-09-14T22:53:02.725962+0000 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:crush_version 5 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:full_ratio 0.95 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:backfillfull_ratio 0.9 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:nearfull_ratio 0.85 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:require_min_compat_client luminous 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:min_compat_client jewel 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:require_osd_release squid 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:stretch_mode_enabled false 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-14T22:53:05.111 INFO:teuthology.orchestra.run.smithi076.stdout:max_osd 8 2024-09-14T22:53:05.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6800/2488220599,v1:172.21.15.76:6801/2488220599] [v2:172.21.15.76:6802/2488220599,v1:172.21.15.76:6803/2488220599] exists,up 45f61e20-37e6-4a13-b005-1f135f5e5a58 2024-09-14T22:53:05.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6808/522829438,v1:172.21.15.76:6809/522829438] [v2:172.21.15.76:6810/522829438,v1:172.21.15.76:6811/522829438] exists,up bc72773d-098f-4ac5-89b5-13dae87b65b4 2024-09-14T22:53:05.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6824/3214755929,v1:172.21.15.76:6825/3214755929] [v2:172.21.15.76:6826/3214755929,v1:172.21.15.76:6827/3214755929] exists,up 2c2a50b0-e6ab-4593-9a26-badf1cc3dd34 2024-09-14T22:53:05.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6816/2699236014,v1:172.21.15.76:6817/2699236014] [v2:172.21.15.76:6818/2699236014,v1:172.21.15.76:6819/2699236014] exists,up 40d20d5f-5509-4a9f-8b13-c9e8da7339bc 2024-09-14T22:53:05.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6824/2097675638,v1:172.21.15.114:6825/2097675638] [v2:172.21.15.114:6826/2097675638,v1:172.21.15.114:6827/2097675638] exists,up 78bf5206-fbf7-4668-a27f-9eb867edc4e5 2024-09-14T22:53:05.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/3469663218,v1:172.21.15.114:6801/3469663218] [v2:172.21.15.114:6802/3469663218,v1:172.21.15.114:6803/3469663218] exists,up 72838af1-36f2-4cf6-a5ee-6daf5d4ea70a 2024-09-14T22:53:05.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/2551471767,v1:172.21.15.114:6817/2551471767] [v2:172.21.15.114:6818/2551471767,v1:172.21.15.114:6819/2551471767] exists,up 11be725a-e81f-4b8a-8a68-acb54e67b443 2024-09-14T22:53:05.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6808/941014947,v1:172.21.15.114:6809/941014947] [v2:172.21.15.114:6810/941014947,v1:172.21.15.114:6811/941014947] exists,up 94103011-716e-495f-aeb3-759071ce6f14 2024-09-14T22:53:05.123 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6800/2488220599,v1:172.21.15.76:6801/2488220599] [v2:172.21.15.76:6802/2488220599,v1:172.21.15.76:6803/2488220599] exists,up 45f61e20-37e6-4a13-b005-1f135f5e5a58', 'osd.1 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6808/522829438,v1:172.21.15.76:6809/522829438] [v2:172.21.15.76:6810/522829438,v1:172.21.15.76:6811/522829438] exists,up bc72773d-098f-4ac5-89b5-13dae87b65b4', 'osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6824/3214755929,v1:172.21.15.76:6825/3214755929] [v2:172.21.15.76:6826/3214755929,v1:172.21.15.76:6827/3214755929] exists,up 2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6816/2699236014,v1:172.21.15.76:6817/2699236014] [v2:172.21.15.76:6818/2699236014,v1:172.21.15.76:6819/2699236014] exists,up 40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'osd.4 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6824/2097675638,v1:172.21.15.114:6825/2097675638] [v2:172.21.15.114:6826/2097675638,v1:172.21.15.114:6827/2097675638] exists,up 78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/3469663218,v1:172.21.15.114:6801/3469663218] [v2:172.21.15.114:6802/3469663218,v1:172.21.15.114:6803/3469663218] exists,up 72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'osd.6 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/2551471767,v1:172.21.15.114:6817/2551471767] [v2:172.21.15.114:6818/2551471767,v1:172.21.15.114:6819/2551471767] exists,up 11be725a-e81f-4b8a-8a68-acb54e67b443', 'osd.7 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6808/941014947,v1:172.21.15.114:6809/941014947] [v2:172.21.15.114:6810/941014947,v1:172.21.15.114:6811/941014947] exists,up 94103011-716e-495f-aeb3-759071ce6f14'] 2024-09-14T22:53:05.124 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T22:53:05.221 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "name": "mon.a", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster": "ceph", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "admin_socket_mode": "", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_client_required": "cephx, none", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_cluster_required": "cephx", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_debug": "true", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_service_required": "cephx", 2024-09-14T22:53:05.222 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_supported": "cephx", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_aio": "true", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_aio_reap_max": "16", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_async_discard": "false", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_block_size": "4096", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_debug_aio": "false", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_enable_discard": "false", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_flock_retry": "3", 2024-09-14T22:53:05.223 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_inject_crash": "0", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_ioring": "false", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_ioring_hipri": "false", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bdev_type": "", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_allocator": "hybrid", 2024-09-14T22:53:05.224 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_buffered_io": "true", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T22:53:05.225 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_replay_recovery": "false", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_sync_write": "false", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_allocator": "hybrid", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T22:53:05.226 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_create": "true", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_db_create": "false", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_db_path": "", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_db_size": "0", 2024-09-14T22:53:05.227 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_path": "", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_size": "107374182400", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_wal_create": "false", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_wal_path": "", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bluefs": "true", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T22:53:05.228 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_autotune": "true", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_size": "0", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T22:53:05.229 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_type": "2q", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_clone_cow": "true", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_mode": "none", 2024-09-14T22:53:05.230 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_freelist": "false", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T22:53:05.231 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T22:53:05.232 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fail_eio": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T22:53:05.233 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_max_blob_size": "0", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T22:53:05.234 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T22:53:05.235 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_spdk_mem": "512", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_tracing": "false", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T22:53:05.236 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T22:53:05.237 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephx_cluster_require_version": "2", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephx_require_signatures": "false", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephx_require_version": "2", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephx_service_require_signatures": "false", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephx_service_require_version": "2", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cephx_sign_messages": "true", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "chdir": "", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_acl_type": "", 2024-09-14T22:53:05.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_asio_thread_count": "2", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_cache_mid": "0.750000", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_cache_size": "16384", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_caps_release_delay": "5", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_check_pool_perm": "true", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_debug_force_sync_read": "false", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_debug_getattr_caps": "false", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_debug_inject_features": "", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_die_on_failed_remount": "false", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_dirsize_rbytes": "true", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_force_lazyio": "false", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_fs": "", 2024-09-14T22:53:05.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_inject_release_failure": "false", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_max_inline_size": "4096", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_mds_namespace": "", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_metadata": "", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_mount_gid": "-1", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_mount_timeout": "300", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_mount_uid": "-1", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_mountpoint": "/", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_notify_timeout": "10", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_oc": "true", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T22:53:05.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_oc_max_objects": "1000", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_oc_size": "209715200", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_permissions": "true", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_quota": "true", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_quota_df": "true", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_readahead_max_bytes": "0", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_readahead_max_periods": "4", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_readahead_min": "131072", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_reconnect_stale": "false", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_shutdown_timeout": "30", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_snapdir": ".snap", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_tick_interval": "1", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_trace": "", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T22:53:05.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_use_faked_inos": "false", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "client_use_random_mds": "false", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "clog_to_graylog": "false", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "clog_to_graylog_port": "12201", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "clog_to_monitors": "default=true", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "clog_to_syslog": "false", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "clog_to_syslog_level": "info", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "-", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_network": "", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_network_interface": "", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "compressor_zlib_isal": "false", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "compressor_zlib_level": "5", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T22:53:05.242 INFO:teuthology.orchestra.run.smithi076.stdout: "compressor_zstd_level": "1", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location": "", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location_hook": "", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_location_hook_timeout": "10", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "daemonize": "false", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "daos_pool": "tank", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T22:53:05.243 INFO:teuthology.orchestra.run.smithi076.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_alienstore": "0/5", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_asok": "1/5", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_asok_assert_abort": "false", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_auth": "20/20", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_bdev": "1/3", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_bluefs": "1/5", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_bluestore": "1/5", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_buffer": "0/1", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T22:53:05.244 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_cephsqlite": "0/5", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_client": "0/5", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_compressor": "1/5", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_context": "0/1", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_crush": "1/1", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_crypto": "1/5", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_cyanstore": "0/5", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_dpdk": "1/5", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_eventtrace": "1/5", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_filer": "0/1", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_filestore": "1/3", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_finisher": "1/1", 2024-09-14T22:53:05.245 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_fuse": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_javaclient": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_journal": "1/3", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_journaler": "0/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_kstore": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_lockdep": "0/1", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mclock": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mds": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mds_balancer": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mds_locker": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mds_log": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mds_migrator": "1/5", 2024-09-14T22:53:05.246 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_memstore": "1/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mgr": "2/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mgrc": "1/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_mon": "20/20", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_monc": "0/10", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_ms": "1/1", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_none": "0/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_objclass": "0/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_objectcacher": "0/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_objecter": "0/1", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_optracker": "0/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_osd": "1/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_paxos": "20/20", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_perfcounter": "1/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_prioritycache": "1/5", 2024-09-14T22:53:05.247 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rados": "0/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rbd": "0/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rbd_replay": "0/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_refs": "0/0", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_reserver": "1/1", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rgw": "1/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rgw_access": "1/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rgw_flight": "1/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rgw_sync": "1/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_rocksdb": "4/5", 2024-09-14T22:53:05.248 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_backref": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_cache": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_device": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_epm": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_journal": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_lba": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_odata": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_omap": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_onode": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_t": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_seastore_tm": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_striper": "0/1", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_test": "0/5", 2024-09-14T22:53:05.249 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_throttle": "1/1", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_timer": "0/1", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_tp": "0/5", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "debug_trace": "1/5", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "device_failure_prediction_mode": "none", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "err_to_graylog": "false", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "err_to_journald": "false", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "err_to_stderr": "true", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "err_to_syslog": "false", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event_tracing": "false", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "exporter_http_port": "9926", 2024-09-14T22:53:05.250 INFO:teuthology.orchestra.run.smithi076.stdout: "exporter_prio_limit": "5", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "exporter_sort_metrics": "true", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "exporter_stats_period": "5", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "fake_statfs_for_testing": "0", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "fatal_signal_handlers": "true", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filer_max_purge_ops": "10", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filer_max_truncate_ops": "128", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_blackhole": "false", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_btrfs_snap": "true", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_caller_concurrency": "10", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T22:53:05.251 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_debug_omap_check": "false", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_debug_verify_split": "false", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_dump_file": "", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_fadvise": "true", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_fail_eio": "true", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_fd_cache_size": "128", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_fiemap": "false", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T22:53:05.252 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_inject_stall": "0", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_journal_parallel": "false", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_journal_trailing": "false", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_journal_writeahead": "false", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_kill_at": "0", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T22:53:05.253 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_merge_threshold": "-10", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_odsync_write": "false", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_omap_backend_path": "", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_op_threads": "2", 2024-09-14T22:53:05.254 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_punch_hole": "false", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_queue_max_ops": "50", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_seek_data_hole": "false", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_sloppy_crc": "false", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T22:53:05.255 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_splice": "false", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_split_multiple": "2", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_split_rand_factor": "20", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_update_to": "1000", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T22:53:05.256 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_xfs_extsize": "false", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "filestore_zfs_snap": "false", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fio_dir": "/tmp/fio", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_allow_other": "true", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_big_writes": "true", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_debug": "false", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_default_permissions": "false", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_disable_pagecache": "false", 2024-09-14T22:53:05.257 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_max_write": "0", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_multithreaded": "true", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_require_active_mds": "true", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_set_user_groups": "true", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_splice_move": "true", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_splice_read": "true", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_splice_write": "true", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "gss_target_name": "ceph", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_file": "", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_inject_failure": "0", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_interval": "5", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "host": "smithi076", 2024-09-14T22:53:05.258 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "inject_early_sigterm": "false", 2024-09-14T22:53:05.259 INFO:teuthology.orchestra.run.smithi076.stdout: "jaeger_agent_port": "6799", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "jaeger_tracing_enable": "false", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_aio": "true", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_align_min_size": "65536", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_block_align": "true", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_block_size": "4096", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_dio": "true", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_discard": "false", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_force_aio": "false", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_ignore_corruption": "false", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_max_write_entries": "100", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_replay_from": "0", 2024-09-14T22:53:05.260 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_write_header_frequency": "0", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "journal_zero_on_create": "false", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "journaler_prefetch_periods": "10", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "journaler_prezero_periods": "5", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "journaler_write_head_interval": "15", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "keyfile": "", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_backend": "rocksdb", 2024-09-14T22:53:05.261 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_max_bytes": "67108864", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_max_ops": "512", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_onode_map_size": "1024", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "kstore_sync_transaction": "false", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "librados_thread_count": "2", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "lockdep": "false", 2024-09-14T22:53:05.262 INFO:teuthology.orchestra.run.smithi076.stdout: "lockdep_force_backtrace": "false", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_coarse_timestamps": "true", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_flush_on_exit": "false", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_graylog_port": "12201", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_max_new": "1000", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_max_recent": "10000", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_stderr_prefix": "", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_file": "true", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_graylog": "false", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_journald": "false", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_stderr": "false", 2024-09-14T22:53:05.263 INFO:teuthology.orchestra.run.smithi076.stdout: "log_to_syslog": "false", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_action_on_write_error": "1", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_alternate_name_max": "8192", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_asio_thread_count": "2", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_export_pin": "true", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_interval": "10", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_max": "-1", 2024-09-14T22:53:05.264 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_max_until": "-1", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_merge_size": "50", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_mode": "0", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_split_bits": "3", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_split_size": "10000", 2024-09-14T22:53:05.265 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_mid": "0.700000", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T22:53:05.266 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_trim_interval": "1", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_debug_auth_pins": "false", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_debug_frag": "false", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_debug_scatterstat": "false", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_debug_subtrees": "false", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T22:53:05.267 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_default_dir_hash": "2", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_defer_session_stale": "true", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dir_max_entries": "0", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dir_prefetch": "true", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_early_reply": "true", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_enable_op_tracker": "true", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_enforce_unique_name": "true", 2024-09-14T22:53:05.268 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T22:53:05.269 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_inject_health_dummy": "false", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_join_fs": "", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_journal_format": "1", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_create_at": "0", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_export_at": "0", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_import_at": "0", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_journal_at": "0", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T22:53:05.270 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_link_at": "0", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_openc_at": "0", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_rename_at": "0", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_max_events": "-1", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_max_segments": "128", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_pause": "false", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_segment_size": "0", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T22:53:05.271 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_trim_threshold": "128", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_completed_requests": "100000", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_export_size": "20971520", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_file_recover": "32", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_purge_files": "64", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_purge_ops": "8192", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_metrics_update_interval": "2", 2024-09-14T22:53:05.272 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_min_caps_per_client": "100", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_numa_node": "-1", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_op_history_duration": "600", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_op_history_size": "20", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_op_log_threshold": "5", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_ping_grace": "15", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_ping_interval": "5", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T22:53:05.273 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_recall_max_caps": "30000", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_replay_interval": "1.000000", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_root_ino_gid": "0", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_root_ino_uid": "0", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T22:53:05.274 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_shutdown_check": "0", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_skip_ino": "0", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_snap_min_uid": "0", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_snap_rstat": "false", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_symlink_recovery": "true", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_thrash_exports": "0", 2024-09-14T22:53:05.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_thrash_fragments": "0", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_tick_interval": "5.000000", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_valgrind_exit": "false", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_verify_backtrace": "1", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_verify_scatter": "false", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_wipe_sessions": "false", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool_debug": "false", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "memstore_page_set": "false", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "memstore_page_size": "65536", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_client_bytes": "134217728", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_client_messages": "512", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T22:53:05.276 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_disabled_modules": "", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_mds_messages": "128", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_mon_messages": "128", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_op_history_duration": "600", 2024-09-14T22:53:05.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_op_history_size": "20", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_op_log_threshold": "5", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_osd_messages": "8192", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_pool": "true", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_standby_modules": "true", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_stats_period": "5", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_stats_threshold": "5", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_tick_period": "2", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_allow_pool_delete": "true", 2024-09-14T22:53:05.278 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_bytes": "104857600", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T22:53:05.279 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_client_target_rank": "-1", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_file_level": "debug", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-14T22:53:05.280 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_compact_on_start": "false", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_compact_on_trim": "true", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_cpu_threads": "4", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_data_avail_crit": "5", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_data_avail_warn": "5", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T22:53:05.281 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_dump_json": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_extra_checks": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_down_uptime_grace": "60", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_election_default_strategy": "1", 2024-09-14T22:53:05.282 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_election_timeout": "5.000000", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_enable_op_tracker": "true", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_fake_pool_delete": "false", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_force_quorum_join": "false", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_health_log_update_period": "5", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_health_max_detail": "50", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_health_to_clog": "true", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T22:53:05.283 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_host": "172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790],172.21.15.114", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_host_override": "", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_initial_members": "", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_lease": "5.000000", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_log_full_interval": "50", 2024-09-14T22:53:05.284 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_log_max": "10000", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_log_max_summary": "50", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_max_log_epochs": "500", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_max_osd": "10000", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_memory_autotune": "true", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_memory_target": "2147483648", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T22:53:05.285 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mgr_digest_period": "5", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_op_complaint_time": "30", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_op_history_duration": "600", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_op_history_size": "20", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_op_log_threshold": "5", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T22:53:05.286 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_cache_size": "500", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T22:53:05.287 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_report_timeout": "900", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T22:53:05.288 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T22:53:05.289 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_reweight_max_osds": "4", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_scrub_interval": "86400", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_scrub_max_keys": "100", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_scrub_timeout": "300", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_session_timeout": "300", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_smart_report_timeout": "5", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T22:53:05.290 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_stretch_pool_size": "4", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_sync_debug": "false", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_tick_interval": "5", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T22:53:05.291 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_older_version": "true", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T22:53:05.292 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "monmap": "", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "motr_tracing_enabled": "false", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_op_threads": "3", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_cm": "false", 2024-09-14T22:53:05.293 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_device_name": "", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_sl": "3", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_rdma_type": "ib", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_async_reap_threshold": "5", 2024-09-14T22:53:05.294 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_before_connect": "false", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_ipv4": "true", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_ipv6": "false", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_msgr1": "true", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_msgr2": "true", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_port_max": "7568", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_port_min": "6800", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_retry_count": "3", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_bind_retry_delay": "5", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_blackhole_client": "false", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_blackhole_mds": "false", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_blackhole_mgr": "false", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_blackhole_mon": "false", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_blackhole_osd": "false", 2024-09-14T22:53:05.295 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_client_mode": "crc secure", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_cluster_type": "", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_compress_secure": "false", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_crc_data": "true", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_crc_header": "true", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_die_on_bug": "true", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_die_on_old_message": "true", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T22:53:05.296 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_hugepages": "", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_lro": "true", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_pmd": "", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_port_id": "0", 2024-09-14T22:53:05.297 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_dump_on_send": "false", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_inject_delay_type": "", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_inject_network_congestion": "0", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_max_accept_failures": "4", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_max_backoff": "15.000000", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T22:53:05.298 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_osd_compress_mode": "none", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_pq_min_cost": "65536", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_public_type": "", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_service_mode": "crc secure", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_tcp_nodelay": "true", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ms_type": "async+posix", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "no_config_file": "false", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T22:53:05.299 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter_inflight_ops": "1024", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter_timeout": "10.000000", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "objectstore_blackhole": "false", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "openssl_engine_opts": "", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T22:53:05.300 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_agent_max_ops": "4", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_agent_slop": "0.020000", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_asio_thread_count": "2", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_backfill_scan_max": "512", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_backfill_scan_min": "64", 2024-09-14T22:53:05.301 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_backoff_on_peering": "false", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_beacon_report_interval": "300", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bench_duration": "30", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_blkin_trace_all": "false", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T22:53:05.302 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_class_update_on_start": "true", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_client_message_cap": "256", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_client_op_priority": "63", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_client_watch_timeout": "30", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_command_max_records": "256", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_compact_on_start": "false", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_crush_update_on_start": "true", 2024-09-14T22:53:05.303 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_op_order": "false", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T22:53:05.304 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_shutdown": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_trim_objects": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T22:53:05.305 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_default_notify_timeout": "30", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_enable_op_tracker": "true", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T22:53:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_fast_info": "true", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_fast_shutdown": "true", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_function_tracing": "false", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_heartbeat_grace": "20", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_heartbeat_interval": "6", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T22:53:05.307 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_heartbeat_stale": "600", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_journal_size": "5120", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_kill_backfill_at": "0", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_map_cache_size": "50", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_map_dedup": "true", 2024-09-14T22:53:05.308 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_map_message_max": "40", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_attr_name_len": "100", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_attr_size": "0", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_backfills": "1", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_markdown_count": "5", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_markdown_period": "600", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_object_name_len": "2048", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_object_size": "134217728", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T22:53:05.309 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_pgls": "1024", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_push_cost": "8388608", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_push_objects": "10", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_scrubs": "3", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_max_write_size": "90", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T22:53:05.310 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_profile": "balanced", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_memory_base": "805306368", 2024-09-14T22:53:05.311 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_memory_target": "4294967296", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_memory_target_autotune": "false", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_min_recovery_priority": "0", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mon_report_interval": "5", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_num_cache_shards": "32", 2024-09-14T22:53:05.312 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_numa_node": "-1", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_objecter_finishers": "1", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_objectstore": "bluestore", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_objectstore_fuse": "false", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_objectstore_tracing": "false", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_history_duration": "600", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_history_size": "20", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_log_threshold": "5", 2024-09-14T22:53:05.313 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_num_shards": "0", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_op_thread_timeout": "15", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_open_classes_on_start": "true", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_os_flags": "0", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_peering_op_priority": "255", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T22:53:05.314 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_delete_priority": "5", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T22:53:05.315 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_crimson": "false", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_flags": "0", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_min_size": "0", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T22:53:05.316 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_size": "2", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_default_type": "replicated", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T22:53:05.317 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_cost": "20971520", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_max_active": "0", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_op_priority": "3", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_priority": "5", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T22:53:05.318 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_repair_during_recovery": "false", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_cost": "52428800", 2024-09-14T22:53:05.319 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_end_hour": "0", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_priority": "5", 2024-09-14T22:53:05.320 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_skip_data_digest": "false", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_smart_report_timeout": "5", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_snap_trim_priority": "5", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T22:53:05.321 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_target_transaction_size": "30", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_tracing": "false", 2024-09-14T22:53:05.322 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_use_stale_snap": "false", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_kill_at": "0", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_max_join_drift": "10", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_min": "500", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_min_wait": "0.050000", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_service_trim_max": "500", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_service_trim_min": "250", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_stash_full_interval": "25", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_trim_max": "500", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "paxos_trim_min": "250", 2024-09-14T22:53:05.323 INFO:teuthology.orchestra.run.smithi076.stdout: "perf": "true", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "-", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrv": "", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "public_bind_addr": "-", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "public_network": "", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "public_network_interface": "", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "qat_compressor_enabled": "false", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "rados_mon_op_timeout": "0", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "rados_osd_op_timeout": "0", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "rados_tracing": "false", 2024-09-14T22:53:05.324 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_atime_update_interval": "60", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache": "true", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache_policy": "writearound", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache_size": "33554432", 2024-09-14T22:53:05.325 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_compression_hint": "none", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_clone_format": "auto", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_data_pool": "", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_features": "61", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_format": "2", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_map_options": "", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_order": "22", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_pool": "rbd", 2024-09-14T22:53:05.326 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_stripe_count": "0", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_io_scheduler": "simple", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T22:53:05.327 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_order": "24", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_pool": "", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_journal_splay_width": "4", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T22:53:05.328 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T22:53:05.329 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T22:53:05.330 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_op_threads": "1", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_plugins": "", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T22:53:05.331 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T22:53:05.332 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_tracing": "false", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_validate_names": "true", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd_validate_pool": "true", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "restapi_base_url": "", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "restapi_log_level": "", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T22:53:05.333 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_admin_entry": "admin", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_backend_store": "rados", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_barbican_url": "", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_beast_enable_async": "true", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T22:53:05.334 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_cache_enabled": "true", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_config_store": "rados", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_content_length_compat": "false", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_cross_domain_policy": "", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T22:53:05.335 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T22:53:05.336 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T22:53:05.337 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T22:53:05.338 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_d4n_port": "6379", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_data_log_window": "30", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T22:53:05.339 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T22:53:05.340 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dns_name": "", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dns_s3website_name": "", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_enable_ops_log": "false", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T22:53:05.341 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_enable_static_website": "false", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_enable_usage_log": "false", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_expose_bucket": "false", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_extended_http_attrs": "", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_filter": "none", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_max_objs": "32", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T22:53:05.342 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_init_timeout": "300", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T22:53:05.343 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_admin_password": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_admin_project": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_admin_token": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_admin_user": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_api_version": "2", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T22:53:05.344 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_url": "", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lc_max_objs": "32", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lc_max_worker": "3", 2024-09-14T22:53:05.345 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ldap_uri": "", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_log_http_headers": "", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T22:53:05.346 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_attr_size": "0", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_listing_results": "1000", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_notify_retries": "10", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T22:53:05.347 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T22:53:05.348 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_num_control_oids": "8", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_numa_node": "-1", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T22:53:05.349 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_op_tracing": "false", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_opa_token": "", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_opa_url": "", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ops_log_rados": "false", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T22:53:05.350 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_print_continue": "true", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T22:53:05.351 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_rados_tracing": "false", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_realm": "", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_realm_id": "", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_region": "", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_request_uri": "", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T22:53:05.352 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_resolve_cname": "false", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_run_sync_thread": "true", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T22:53:05.353 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_script_uri": "", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_service_provider_name": "", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sts_client_id": "", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sts_client_secret": "", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sts_entry": "sts", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sts_key": "sts", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_auth_url": "", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_custom_header": "", 2024-09-14T22:53:05.354 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_need_stats": "true", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_tenant_name": "", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_url": "", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_lease_period": "120", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T22:53:05.355 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_thread_pool_size": "512", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_torrent_comment": "", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_torrent_createby": "", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_torrent_encoding": "", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_torrent_flag": "false", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T22:53:05.356 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_torrent_origin": "", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_torrent_tracker": "", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_usage_max_shards": "32", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_use_opa_authz": "false", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_counters_cache": "false", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T22:53:05.357 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_user_unique_email": "true", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_verify_ssl": "true", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_zone": "", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_zone_id": "", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_zonegroup": "", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_zonegroup_id": "", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T22:53:05.358 INFO:teuthology.orchestra.run.smithi076.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_block_size": "4096", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T22:53:05.359 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_partition_filters": "false", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_perf": "false", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "run_dir": "/var/run/ceph", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_block_create": "true", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T22:53:05.360 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_device_size": "53687091200", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_full_integrity_check": "false", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_main_device_type": "SSD", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T22:53:05.361 INFO:teuthology.orchestra.run.smithi076.stdout: "seastore_segment_size": "67108864", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "set_keepcaps": "false", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "setgroup": "", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "setuser": "", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "setuser_match_path": "", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "thp": "false", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "threadpool_default_timeout": "60", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout: "throttler_perf_counter": "true" 2024-09-14T22:53:05.362 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:05.364 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T22:53:05.476 INFO:teuthology.orchestra.run.smithi076.stdout:mon.a: {} 2024-09-14T22:53:05.476 INFO:teuthology.orchestra.run.smithi076.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:53:05.486 INFO:teuthology.orchestra.run.smithi076.stdout:mon.b: {} 2024-09-14T22:53:05.486 INFO:teuthology.orchestra.run.smithi076.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:53:05.494 INFO:teuthology.orchestra.run.smithi076.stdout:mon.c: {} 2024-09-14T22:53:05.495 INFO:teuthology.orchestra.run.smithi076.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:53:05.505 DEBUG:teuthology.orchestra.run.smithi076:> type ceph-objectstore-tool 2024-09-14T22:53:05.519 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T22:53:05.519 DEBUG:teuthology.orchestra.run.smithi114:> type ceph-objectstore-tool 2024-09-14T22:53:05.537 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T22:53:05.538 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T22:53:05.548 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T22:53:05.548 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T22:53:05.548 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:05.548 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:05.548 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T22:53:05.549 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:53:05.549 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T22:53:05.549 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T22:53:05.549 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:05.550 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T22:53:05.550 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:05.550 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T22:53:05.550 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:53:05.551 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-14T22:53:05.579 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:05.577+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:05.598 INFO:teuthology.orchestra.run.smithi076.stdout:mapped writes DISABLED 2024-09-14T22:53:05.598 INFO:teuthology.orchestra.run.smithi076.stdout:mapped reads DISABLED 2024-09-14T22:53:05.598 INFO:teuthology.orchestra.run.smithi076.stdout:Seed set to 3613 2024-09-14T22:53:05.651 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:05.648+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:05.678 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:05.679 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:05.679 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:05.679 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:05.692 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:05.702 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:05.703 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:05.722 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:05.723 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:05.741 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:05.741 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:05.751 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:05.749+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:05.764 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:05.764 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:05.781 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:05.781 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:05.798 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:05.799 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:05.815 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:05.815 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:05.817 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:05.817 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:05.826 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:05.850 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:05.850 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:05.853 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:05.851+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:05.907 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:05.907 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:05.907 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 24, 2024-09-14T22:53:05.907 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:53:05.734283+0000", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5799936, 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:53:05.908 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "jewel", 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:53:05.909 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:05.910 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:53:05.911 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:05.912 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.913 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:53:05.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:05.915 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:05.916 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:05.917 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:05.918 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:05.919 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:05.920 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.9900000095367432, 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.9900000095367432, 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.75, 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.75, 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:05.921 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 32769, 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,creating", 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:05.922 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:05.923 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "24", 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:05.924 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:05.925 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:05.926 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": {}, 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:05.927 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.5, 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.5, 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.5, 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.5, 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:05.929 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 0, 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:53:05.930 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.931 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.932 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:05.934 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 19, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:05.935 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.938 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:53:05.939 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:05.940 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 19, 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.942 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.943 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.944 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:05.945 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 0, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:05.946 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.949 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.950 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:05.951 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 0, 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:05.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:05.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:05.956 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 19, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:05.957 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.959 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:53:05.961 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 19, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.964 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.965 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.966 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:53:05.967 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:05.968 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 19, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:05.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:53:05.971 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:53:05.972 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:05.973 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:05.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:05.975 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:53:05.976 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:05.978 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:53:05.979 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [], 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [], 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [], 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:53:05.980 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [], 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:53:05.981 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:53:05.982 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:53:05.982 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:53:05.982 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:05.982 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:05.984 INFO:tasks.thrashosds.thrasher:j is {'epoch': 24, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:53:05.734283+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:53:05.984 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:53:05.985 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:05.985 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:05.985 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:05.985 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:05.981+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:06.083 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:06.083+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:06.130 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:06.130 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:06.139 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:06.185 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:06.183+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:06.244 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:06.244 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:06.253 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:06.285 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:06.284+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:06.309 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:06.310 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:53:06.322 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-14T22:53:05.743761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219248,"kb_used_data":3024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755472,"statfs":{"total":773094113280,"available":772869603328,"internally_reserved":0,"allocated":3096576,"data_stored":1838278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.960417"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.805320+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:53:02.805320+0000","last_peered":"2024-09-14T22:53:02.805320+0000","last_clean":"2024-09-14T22:53:02.805320+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:53:02.805320+0000","last_undegraded":"2024-09-14T22:53:02.805320+0000","last_fullsized":"2024-09-14T22:53:02.805320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.806143+0000","last_change":"2024-09-14T22:52:59.538561+0000","last_active":"2024-09-14T22:53:02.806143+0000","last_peered":"2024-09-14T22:53:02.806143+0000","last_clean":"2024-09-14T22:53:02.806143+0000","last_became_active":"2024-09-14T22:52:57.530436+0000","last_became_peered":"2024-09-14T22:52:57.530436+0000","last_unstale":"2024-09-14T22:53:02.806143+0000","last_undegraded":"2024-09-14T22:53:02.806143+0000","last_fullsized":"2024-09-14T22:53:02.806143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:29.562282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.804672+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:02.804672+0000","last_peered":"2024-09-14T22:53:02.804672+0000","last_clean":"2024-09-14T22:53:02.804672+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:02.804672+0000","last_undegraded":"2024-09-14T22:53:02.804672+0000","last_fullsized":"2024-09-14T22:53:02.804672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.805921+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:53:02.805921+0000","last_peered":"2024-09-14T22:53:02.805921+0000","last_clean":"2024-09-14T22:53:02.805921+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:53:02.805921+0000","last_undegraded":"2024-09-14T22:53:02.805921+0000","last_fullsized":"2024-09-14T22:53:02.805921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.805429+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:53:02.805429+0000","last_peered":"2024-09-14T22:53:02.805429+0000","last_clean":"2024-09-14T22:53:02.805429+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:53:02.805429+0000","last_undegraded":"2024-09-14T22:53:02.805429+0000","last_fullsized":"2024-09-14T22:53:02.805429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.808492+0000","last_change":"2024-09-14T22:53:00.039871+0000","last_active":"2024-09-14T22:53:02.808492+0000","last_peered":"2024-09-14T22:53:02.808492+0000","last_clean":"2024-09-14T22:53:02.808492+0000","last_became_active":"2024-09-14T22:52:57.694658+0000","last_became_peered":"2024-09-14T22:52:57.694658+0000","last_unstale":"2024-09-14T22:53:02.808492+0000","last_undegraded":"2024-09-14T22:53:02.808492+0000","last_fullsized":"2024-09-14T22:53:02.808492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:09.229943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.804650+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:02.804650+0000","last_peered":"2024-09-14T22:53:02.804650+0000","last_clean":"2024-09-14T22:53:02.804650+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:02.804650+0000","last_undegraded":"2024-09-14T22:53:02.804650+0000","last_fullsized":"2024-09-14T22:53:02.804650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.805458+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:53:02.805458+0000","last_peered":"2024-09-14T22:53:02.805458+0000","last_clean":"2024-09-14T22:53:02.805458+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:53:02.805458+0000","last_undegraded":"2024-09-14T22:53:02.805458+0000","last_fullsized":"2024-09-14T22:53:02.805458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":172,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T22:53:02.804612+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:02.804612+0000","last_peered":"2024-09-14T22:53:02.804612+0000","last_clean":"2024-09-14T22:53:02.804612+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:02.804612+0000","last_undegraded":"2024-09-14T22:53:02.804612+0000","last_fullsized":"2024-09-14T22:53:02.804612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T22:53:05.734283+0000","last_change":"2024-09-14T22:53:05.734283+0000","last_active":"2024-09-14T22:53:05.734283+0000","last_peered":"2024-09-14T22:53:05.734283+0000","last_clean":"2024-09-14T22:53:05.734283+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:53:05.734283+0000","last_undegraded":"2024-09-14T22:53:05.734283+0000","last_fullsized":"2024-09-14T22:53:05.734283+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":82188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":82188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":82188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":82188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:53:06.325 INFO:tasks.thrashosds.thrasher:Setting 3.0 to [6, 7] 2024-09-14T22:53:06.325 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.0', '6', '7'] 2024-09-14T22:53:06.326 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.0 6 7 2024-09-14T22:53:06.358 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:06.358 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:06.368 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:06.409 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:06.407+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:06.487 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:06.484+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:06.514 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:06.515 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:06.524 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:06.588 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:06.586+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:06.628 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:06.628 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:06.636 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:06.689 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:06.686+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:06.740 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:06.740 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:06.745 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:53:06.749 INFO:teuthology.orchestra.run.smithi076.stderr:set 3.0 pg_upmap mapping to [6,7] 2024-09-14T22:53:06.751 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:06.816 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:06.814+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:06.876 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:06.876 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:06.884 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:06.925 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:06.923+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:06.991 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:06.989+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:07.028 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:07.028 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:07.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:07.092 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:07.090+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:07.140 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:07.140 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:07.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:07.193 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:07.191+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:07.264 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:07.264 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:07.273 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:07.294 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:07.291+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:07.394 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:07.392+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:07.417 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:07.417 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:07.425 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:07.495 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:07.494+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:07.569 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:07.569 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:07.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:07.621 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:07.619+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:07.680 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:07.680 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:07.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:07.698 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:07.696+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:07.799 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:07.797+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:07.833 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:07.833 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:07.841 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:07.862 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:07.926 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:07.924+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:07.985 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:07.985 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:07.987 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:07.987 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:07.994 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:08.001 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:07.999+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:08.004 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:08.004 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:08.020 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:08.020 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:08.036 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:08.036 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:08.053 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:08.054 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:08.070 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:08.070 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:08.086 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:08.086 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:08.102 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:08.102 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:08.102 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:08.100+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:08.139 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:08.140 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:08.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:08.233 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:08.231+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:08.292 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:08.292 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:08.300 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:08.341 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:08.339+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:08.404 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:08.402+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:08.443 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [] 2024-09-14T22:53:08.443 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:08.451 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:08.505 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:08.503+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:08.594 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:08.594 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:08.602 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:08.605 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:08.604+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:08.705 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:08.706 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:08.706 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:08.703+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:08.714 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:08.768 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:53:08.807 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:08.804+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.353350203000002, 2024-09-14T22:53:08.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:08.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:53:08.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.693400+0000", 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.167725700999998, 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076328840000000004, 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 37 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.693400+0000", 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.693400+0000", 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.693400+0000", 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1299999999999999e-07 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.693410+0000", 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5740000000000002e-06 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.693414+0000", 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1559999999999997e-06 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.693417+0000", 2024-09-14T22:53:08.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0979999999999998e-06 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.697746+0000", 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043286449999999999 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.697801+0000", 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4812999999999998e-05 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.697887+0000", 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6420000000000003e-05 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.698636+0000", 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00074877500000000003 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.700958+0000", 2024-09-14T22:53:08.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002322574 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.700994+0000", 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5407000000000002e-05 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.701033+0000", 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9208999999999997e-05 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.738989+0000", 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.122136556000001, 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096166510000000004, 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 70 2024-09-14T22:53:08.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.738989+0000", 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.738989+0000", 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.738990+0000", 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.738994+0000", 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3290000000000004e-06 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.739000+0000", 2024-09-14T22:53:08.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3229999999999997e-06 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.739003+0000", 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3299999999999999e-06 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.743306+0000", 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043028570000000002 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.743383+0000", 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7232999999999997e-05 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.743510+0000", 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126687 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.744710+0000", 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001200504 2024-09-14T22:53:08.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.748533+0000", 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003822715 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.748566+0000", 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3485999999999998e-05 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.748606+0000", 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9524000000000003e-05 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.983180+0000", 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.877945592, 2024-09-14T22:53:08.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071011980000000004, 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 114 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983180+0000", 2024-09-14T22:53:08.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983180+0000", 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983181+0000", 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8400000000000005e-07 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983184+0000", 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6739999999999999e-06 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983192+0000", 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1319999999999996e-06 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983198+0000", 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.173e-06 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.988366+0000", 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051681679999999999 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.988416+0000", 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9749000000000001e-05 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.988512+0000", 2024-09-14T22:53:08.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6366999999999994e-05 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.989868+0000", 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013562769999999999 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990221+0000", 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035306500000000001 2024-09-14T22:53:08.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990249+0000", 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8067000000000001e-05 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990281+0000", 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2042000000000003e-05 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.891 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.891 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.983205+0000", 2024-09-14T22:53:08.891 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.877920634000001, 2024-09-14T22:53:08.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071801740000000001, 2024-09-14T22:53:08.891 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.892 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.892 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.892 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.892 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.892 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.892 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 115 2024-09-14T22:53:08.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.893 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983205+0000", 2024-09-14T22:53:08.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983205+0000", 2024-09-14T22:53:08.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983206+0000", 2024-09-14T22:53:08.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2899999999999999e-07 2024-09-14T22:53:08.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983260+0000", 2024-09-14T22:53:08.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4527999999999997e-05 2024-09-14T22:53:08.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983268+0000", 2024-09-14T22:53:08.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8499999999999994e-06 2024-09-14T22:53:08.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983276+0000", 2024-09-14T22:53:08.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2770000000000005e-06 2024-09-14T22:53:08.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.988786+0000", 2024-09-14T22:53:08.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055099950000000002 2024-09-14T22:53:08.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.988835+0000", 2024-09-14T22:53:08.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8384999999999998e-05 2024-09-14T22:53:08.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.988922+0000", 2024-09-14T22:53:08.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7113000000000004e-05 2024-09-14T22:53:08.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990289+0000", 2024-09-14T22:53:08.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001367633 2024-09-14T22:53:08.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990326+0000", 2024-09-14T22:53:08.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6526999999999998e-05 2024-09-14T22:53:08.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990362+0000", 2024-09-14T22:53:08.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6346000000000001e-05 2024-09-14T22:53:08.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990385+0000", 2024-09-14T22:53:08.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3091e-05 2024-09-14T22:53:08.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.904 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.905 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.905 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.905 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.983287+0000", 2024-09-14T22:53:08.906 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.8778392, 2024-09-14T22:53:08.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072274369999999998, 2024-09-14T22:53:08.906 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.906 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.906 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.907 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.907 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.907 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116 2024-09-14T22:53:08.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.907 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983287+0000", 2024-09-14T22:53:08.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983287+0000", 2024-09-14T22:53:08.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983287+0000", 2024-09-14T22:53:08.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1399999999999997e-07 2024-09-14T22:53:08.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983292+0000", 2024-09-14T22:53:08.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3780000000000003e-06 2024-09-14T22:53:08.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983299+0000", 2024-09-14T22:53:08.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8870000000000003e-06 2024-09-14T22:53:08.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983303+0000", 2024-09-14T22:53:08.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8770000000000001e-06 2024-09-14T22:53:08.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.989511+0000", 2024-09-14T22:53:08.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062073919999999999 2024-09-14T22:53:08.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.989574+0000", 2024-09-14T22:53:08.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3374999999999995e-05 2024-09-14T22:53:08.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.989663+0000", 2024-09-14T22:53:08.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9248999999999999e-05 2024-09-14T22:53:08.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990295+0000", 2024-09-14T22:53:08.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00063185300000000004 2024-09-14T22:53:08.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990467+0000", 2024-09-14T22:53:08.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017197600000000001 2024-09-14T22:53:08.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990494+0000", 2024-09-14T22:53:08.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6826e-05 2024-09-14T22:53:08.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990514+0000", 2024-09-14T22:53:08.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.001e-05 2024-09-14T22:53:08.918 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.983312+0000", 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.877813978000001, 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077904849999999998, 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.919 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 117 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983312+0000", 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983312+0000", 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983312+0000", 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5999999999999999e-07 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983316+0000", 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8480000000000003e-06 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983323+0000", 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0990000000000001e-06 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.983328+0000", 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9649999999999999e-06 2024-09-14T22:53:08.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.989897+0000", 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065690690000000003 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.989957+0000", 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9666000000000001e-05 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990033+0000", 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5605000000000005e-05 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.990795+0000", 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00076241199999999996 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991054+0000", 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025934900000000003 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991081+0000", 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6642999999999999e-05 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991102+0000", 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1369000000000002e-05 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:121 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.991489+0000", 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.869637111999999, 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071340279999999997, 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.926 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 121 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991489+0000", 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991489+0000", 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991489+0000", 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.44e-07 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991558+0000", 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8616999999999997e-05 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991567+0000", 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6610000000000006e-06 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.991576+0000", 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9539999999999993e-06 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.992621+0000", 2024-09-14T22:53:08.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010452809999999999 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.992667+0000", 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5921000000000003e-05 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.992738+0000", 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0788000000000001e-05 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.993969+0000", 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001231734 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.998543+0000", 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045732159999999997 2024-09-14T22:53:08.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.998600+0000", 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7479999999999999e-05 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.998623+0000", 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2731999999999999e-05 2024-09-14T22:53:08.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383208+0000", 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477918036, 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075004249999999998, 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.933 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 131 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383208+0000", 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383208+0000", 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383208+0000", 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5599999999999995e-07 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383298+0000", 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.92e-05 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383304+0000", 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4200000000000004e-06 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383308+0000", 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2969999999999997e-06 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.384026+0000", 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00071793599999999996 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.384074+0000", 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8262e-05 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.384150+0000", 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5710999999999994e-05 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.386730+0000", 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002580036 2024-09-14T22:53:08.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390638+0000", 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039080039999999996 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390669+0000", 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0883999999999998e-05 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390708+0000", 2024-09-14T22:53:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9119000000000003e-05 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383316+0000", 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477810120999999, 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075129180000000004, 2024-09-14T22:53:08.940 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 132 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383316+0000", 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383316+0000", 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383316+0000", 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.84e-07 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383318+0000", 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2740000000000002e-06 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383322+0000", 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9469999999999996e-06 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383327+0000", 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5349999999999998e-06 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.384589+0000", 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0012621970000000001 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.384642+0000", 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2555000000000001e-05 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.384717+0000", 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5604000000000003e-05 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.386737+0000", 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020202559999999998 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390752+0000", 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040141689999999997 2024-09-14T22:53:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390789+0000", 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7780000000000001e-05 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390829+0000", 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9217000000000001e-05 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383437+0000", 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477689269000001, 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075254090000000003, 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.947 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 133 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383437+0000", 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383437+0000", 2024-09-14T22:53:08.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383437+0000", 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1900000000000002e-07 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383440+0000", 2024-09-14T22:53:08.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.095e-06 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383451+0000", 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0635000000000001e-05 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383456+0000", 2024-09-14T22:53:08.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8189999999999999e-06 2024-09-14T22:53:08.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.385015+0000", 2024-09-14T22:53:08.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015593569999999999 2024-09-14T22:53:08.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.385103+0000", 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7571000000000006e-05 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.385239+0000", 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013638800000000001 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.387324+0000", 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020852850000000001 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390876+0000", 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035519470000000002 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390920+0000", 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4141000000000003e-05 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390962+0000", 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1551999999999998e-05 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.954 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383504+0000", 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477621871, 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075550249999999999, 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 134 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383504+0000", 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383504+0000", 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383505+0000", 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-07 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383547+0000", 2024-09-14T22:53:08.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2759000000000002e-05 2024-09-14T22:53:08.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383555+0000", 2024-09-14T22:53:08.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.106e-06 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383561+0000", 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.913e-06 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.385638+0000", 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020769410000000001 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.385685+0000", 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6891999999999998e-05 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.385758+0000", 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2673e-05 2024-09-14T22:53:08.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.387332+0000", 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00157362 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391006+0000", 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036748900000000001 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391036+0000", 2024-09-14T22:53:08.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9090000000000001e-05 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391059+0000", 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3476e-05 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383578+0000", 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477548093999999, 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075829269999999997, 2024-09-14T22:53:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 135 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383578+0000", 2024-09-14T22:53:08.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383578+0000", 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383578+0000", 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7999999999999995e-07 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383582+0000", 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6389999999999998e-06 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383590+0000", 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2989999999999997e-06 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383597+0000", 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2990000000000002e-06 2024-09-14T22:53:08.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.386186+0000", 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0025894109999999998 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.386262+0000", 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6157999999999998e-05 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.386393+0000", 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013032399999999999 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388111+0000", 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001718591 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391101+0000", 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029901139999999999 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391138+0000", 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6715999999999999e-05 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391161+0000", 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2696000000000001e-05 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383607+0000", 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477518774, 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007639196, 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 136 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383607+0000", 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383607+0000", 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383608+0000", 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9599999999999999e-07 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383611+0000", 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7139999999999999e-06 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383618+0000", 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2069999999999998e-06 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383624+0000", 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.386778+0000", 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031543859999999999 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.386827+0000", 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8764999999999998e-05 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.386902+0000", 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5426000000000006e-05 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388885+0000", 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001982382 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391198+0000", 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023131919999999999 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391226+0000", 2024-09-14T22:53:08.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7815999999999999e-05 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391246+0000", 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0571999999999999e-05 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383633+0000", 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477493232, 2024-09-14T22:53:08.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007690082, 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 137 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383633+0000", 2024-09-14T22:53:08.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383633+0000", 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383633+0000", 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1500000000000005e-07 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383683+0000", 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9929000000000003e-05 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383689+0000", 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383693+0000", 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7869999999999999e-06 2024-09-14T22:53:08.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.387478+0000", 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037850319999999998 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.387529+0000", 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0328999999999999e-05 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.387602+0000", 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3918000000000006e-05 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390601+0000", 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029982630000000001 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391276+0000", 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000675108 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391303+0000", 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7035000000000001e-05 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391323+0000", 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9775999999999998e-05 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383701+0000", 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477424958, 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077022569999999997, 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 138 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383701+0000", 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383701+0000", 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383701+0000", 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.08e-07 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383704+0000", 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.295e-06 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383707+0000", 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6009999999999999e-06 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383710+0000", 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7159999999999999e-06 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388145+0000", 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044347809999999996 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388193+0000", 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8520000000000003e-05 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388269+0000", 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5975000000000004e-05 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390836+0000", 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0025672580000000002 2024-09-14T22:53:08.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391356+0000", 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000519225 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391383+0000", 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6885000000000001e-05 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391403+0000", 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0593000000000001e-05 2024-09-14T22:53:08.989 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383714+0000", 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477412299999999, 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077633049999999999, 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.990 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 139 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383714+0000", 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383714+0000", 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383714+0000", 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3999999999999998e-07 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383716+0000", 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8929999999999999e-06 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383721+0000", 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1970000000000004e-06 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383729+0000", 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6529999999999994e-06 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:08.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388541+0000", 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048128119999999996 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388589+0000", 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7441999999999997e-05 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:08.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388663+0000", 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4510000000000003e-05 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390839+0000", 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002175639 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391434+0000", 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059473499999999999 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391459+0000", 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5043999999999999e-05 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391477+0000", 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8139999999999999e-05 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383736+0000", 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477389835, 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078213669999999992, 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 140 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383736+0000", 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383736+0000", 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383736+0000", 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.65e-07 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383767+0000", 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0660000000000001e-05 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383771+0000", 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1629999999999998e-06 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383775+0000", 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5499999999999999e-06 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:08.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388921+0000", 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051461160000000001 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.388976+0000", 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5609999999999998e-05 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.389054+0000", 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7588000000000003e-05 2024-09-14T22:53:09.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391066+0000", 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020121319999999998 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391510+0000", 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044415399999999999 2024-09-14T22:53:09.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391536+0000", 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6078999999999999e-05 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391557+0000", 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.105e-05 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.383781+0000", 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.477344328999999, 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078382599999999997, 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 141 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:09.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:09.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383781+0000", 2024-09-14T22:53:09.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:09.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:09.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383781+0000", 2024-09-14T22:53:09.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383782+0000", 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1100000000000002e-07 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383784+0000", 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1619999999999998e-06 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383788+0000", 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3470000000000001e-06 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.383791+0000", 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9069999999999999e-06 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390254+0000", 2024-09-14T22:53:09.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064631929999999999 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390309+0000", 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4945999999999999e-05 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.390385+0000", 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5325000000000001e-05 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391409+0000", 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010246560000000001 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391578+0000", 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016821100000000001 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391601+0000", 2024-09-14T22:53:09.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3541999999999999e-05 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.391620+0000", 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8660000000000001e-05 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:53.843573+0000", 2024-09-14T22:53:09.009 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.017552452, 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069841670000000003, 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 159 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:09.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.843573+0000", 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.843573+0000", 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.843574+0000", 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1e-07 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.843576+0000", 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2350000000000002e-06 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.843580+0000", 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.34e-06 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.843583+0000", 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.818e-06 2024-09-14T22:53:09.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.847099+0000", 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035158889999999999 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.847173+0000", 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3547000000000006e-05 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.847292+0000", 2024-09-14T22:53:09.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011937 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.848096+0000", 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00080375399999999995 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.850478+0000", 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023828529999999999 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.850524+0000", 2024-09-14T22:53:09.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5512000000000001e-05 2024-09-14T22:53:09.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:09.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:09.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:09.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:53.850558+0000", 2024-09-14T22:53:09.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3538999999999999e-05 2024-09-14T22:53:09.015 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:09.015 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:09.015 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:09.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:09.016 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:09.016 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:09.017 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:09.018 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:09.014+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:09.117 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:09.115+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:09.122 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:09.122 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:09.122 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:09.122 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:09.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:09.219 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:09.216+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:09.277 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:09.277 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:09.277 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:09.277 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:09.277 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:09.286 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:09.319 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:09.318+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:09.420 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:09.419+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:09.433 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:09.433 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:09.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.654469+0000", 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.776637220000001, 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0044819769999999998, 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654469+0000", 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654469+0000", 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654470+0000", 2024-09-14T22:53:09.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654478+0000", 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5120000000000008e-06 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654487+0000", 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0890000000000006e-06 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654495+0000", 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4499999999999998e-06 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654543+0000", 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8128999999999999e-05 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654568+0000", 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5077999999999999e-05 2024-09-14T22:53:09.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.658913+0000", 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043456370000000003 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.658951+0000", 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7446e-05 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:18 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'8, mlcod=17'5)", 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.674322+0000", 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.756784226000001, 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0040224830000000003, 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674322+0000", 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674322+0000", 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674323+0000", 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5600000000000004e-07 2024-09-14T22:53:09.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674339+0000", 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6702999999999999e-05 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674353+0000", 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3286e-05 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674358+0000", 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5779999999999998e-06 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674412+0000", 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3434999999999999e-05 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674428+0000", 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6073e-05 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.678303+0000", 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0038748609999999998 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.678344+0000", 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1690999999999998e-05 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.683550+0000", 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.74755575, 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032167889999999998, 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683550+0000", 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683550+0000", 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683551+0000", 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0200000000000001e-07 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683557+0000", 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4180000000000002e-06 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683565+0000", 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8329999999999994e-06 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683571+0000", 2024-09-14T22:53:09.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4070000000000001e-06 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683616+0000", 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5170999999999998e-05 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683637+0000", 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0761e-05 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.686738+0000", 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003101159 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.686767+0000", 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9337999999999999e-05 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:120 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'82, mlcod=17'78)", 2024-09-14T22:53:09.448 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.992413+0000", 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.438692824, 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0029939319999999999, 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992413+0000", 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992413+0000", 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992414+0000", 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6799999999999999e-07 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992460+0000", 2024-09-14T22:53:09.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6294000000000001e-05 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992472+0000", 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1771e-05 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992477+0000", 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1719999999999998e-06 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992512+0000", 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4499999999999998e-05 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992525+0000", 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3366e-05 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.995372+0000", 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002846389 2024-09-14T22:53:09.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.995407+0000", 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5571999999999998e-05 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:121 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'83, mlcod=17'78)", 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.992798+0000", 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.438307849000001, 2024-09-14T22:53:09.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054025760000000001, 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992798+0000", 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992798+0000", 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992799+0000", 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7199999999999999e-07 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992872+0000", 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2799999999999994e-05 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992882+0000", 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.025e-05 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992889+0000", 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7129999999999999e-06 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992924+0000", 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5510999999999999e-05 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992937+0000", 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2490000000000001e-05 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.998170+0000", 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052333060000000001 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.998201+0000", 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0933999999999999e-05 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:129 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'87, mlcod=17'85)", 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.383413+0000", 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.047693263999999, 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031314799999999999, 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.458 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383413+0000", 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383413+0000", 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383414+0000", 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4700000000000001e-07 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383419+0000", 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1560000000000003e-06 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383429+0000", 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0757000000000001e-05 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383434+0000", 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8960000000000002e-06 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383467+0000", 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2546000000000003e-05 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383479+0000", 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1987000000000001e-05 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386488+0000", 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030093329999999999 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386544+0000", 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6158e-05 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:130 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'88, mlcod=17'85)", 2024-09-14T22:53:09.462 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.383847+0000", 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.047259560000001, 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030430019999999999, 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383847+0000", 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383847+0000", 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383847+0000", 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383852+0000", 2024-09-14T22:53:09.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7480000000000001e-06 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383862+0000", 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8079999999999999e-06 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383866+0000", 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6319999999999999e-06 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383913+0000", 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7092999999999998e-05 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383931+0000", 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7428999999999999e-05 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386864+0000", 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0029328290000000001 2024-09-14T22:53:09.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386890+0000", 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5876000000000001e-05 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:131 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'89, mlcod=17'85)", 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.384287+0000", 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.046819244999998, 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054347629999999996, 2024-09-14T22:53:09.467 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384287+0000", 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384287+0000", 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384287+0000", 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384313+0000", 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5525e-05 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384323+0000", 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0379e-05 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384328+0000", 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7060000000000003e-06 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384364+0000", 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6229999999999997e-05 2024-09-14T22:53:09.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384375+0000", 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1080000000000001e-05 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389686+0000", 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053104720000000001 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389722+0000", 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5790999999999997e-05 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:132 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'90, mlcod=17'85)", 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.384792+0000", 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.046313731000001, 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0049634750000000002, 2024-09-14T22:53:09.472 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384792+0000", 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384792+0000", 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384793+0000", 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.82e-07 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384798+0000", 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5249999999999999e-06 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384808+0000", 2024-09-14T22:53:09.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0200999999999999e-05 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384812+0000", 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384848+0000", 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5438999999999997e-05 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384860+0000", 2024-09-14T22:53:09.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.201e-05 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389725+0000", 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048650940000000004 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389756+0000", 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0942000000000003e-05 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:133 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'91, mlcod=17'85)", 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.385363+0000", 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.045742707999999, 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00442678, 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385363+0000", 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385363+0000", 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385364+0000", 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5000000000000002e-07 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385369+0000", 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7890000000000002e-06 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385379+0000", 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0540000000000001e-05 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385384+0000", 2024-09-14T22:53:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6569999999999996e-06 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385423+0000", 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8693999999999999e-05 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385434+0000", 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1309e-05 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389757+0000", 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043230609999999996 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389790+0000", 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3080000000000002e-05 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:134 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'92, mlcod=17'85)", 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.385819+0000", 2024-09-14T22:53:09.481 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.045287317, 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0040019410000000002, 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385819+0000", 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385819+0000", 2024-09-14T22:53:09.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385819+0000", 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1899999999999998e-07 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385883+0000", 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3774000000000003e-05 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385890+0000", 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9349999999999997e-06 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385895+0000", 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2889999999999997e-06 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385929+0000", 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3983999999999999e-05 2024-09-14T22:53:09.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385942+0000", 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3274e-05 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389791+0000", 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0038488060000000002 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389821+0000", 2024-09-14T22:53:09.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9560000000000002e-05 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:135 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'93, mlcod=17'85)", 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.386482+0000", 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.044623937000001, 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0033692790000000002, 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.486 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386482+0000", 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386482+0000", 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386483+0000", 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386488+0000", 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.763e-06 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386500+0000", 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1883e-05 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386509+0000", 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6579999999999997e-06 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386568+0000", 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8477999999999998e-05 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386581+0000", 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3808999999999999e-05 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389823+0000", 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003241318 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389851+0000", 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8685999999999999e-05 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.490 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:136 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'94, mlcod=17'85)", 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.386959+0000", 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.044147573, 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0029261230000000001, 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386959+0000", 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386959+0000", 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386959+0000", 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.99e-07 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386963+0000", 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3390000000000003e-06 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386971+0000", 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6429999999999995e-06 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386977+0000", 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.891e-06 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.387008+0000", 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1343999999999997e-05 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.387019+0000", 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0633e-05 2024-09-14T22:53:09.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389853+0000", 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0028337729999999999 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389885+0000", 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1900999999999998e-05 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.495 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'98)", 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.393598+0000", 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.037508377000002, 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0024712050000000002, 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393598+0000", 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393598+0000", 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393598+0000", 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393664+0000", 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5845000000000004e-05 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393671+0000", 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1930000000000004e-06 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393675+0000", 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7249999999999999e-06 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393707+0000", 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2224999999999997e-05 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393720+0000", 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3132999999999999e-05 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.396038+0000", 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0023174100000000002 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.396069+0000", 2024-09-14T22:53:09.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1327e-05 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:155 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=17'105)", 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.844740+0000", 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 15.586366482000001, 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002873914, 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.500 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844740+0000", 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844740+0000", 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844740+0000", 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4999999999999999e-07 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844819+0000", 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9184999999999995e-05 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844827+0000", 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6129999999999998e-06 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844832+0000", 2024-09-14T22:53:09.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4140000000000002e-06 2024-09-14T22:53:09.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844862+0000", 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9774e-05 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844876+0000", 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3794e-05 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847572+0000", 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0026964110000000001 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847614+0000", 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1473000000000001e-05 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:156 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'110, mlcod=17'105)", 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.845407+0000", 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 15.585699523000001, 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0046289319999999997, 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.505 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845407+0000", 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845407+0000", 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845407+0000", 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:53:09.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845412+0000", 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4830000000000001e-06 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845419+0000", 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.956e-06 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845425+0000", 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:53:09.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845456+0000", 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0837000000000003e-05 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845467+0000", 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1419e-05 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.849994+0000", 2024-09-14T22:53:09.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0045264190000000003 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850036+0000", 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1757e-05 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:157 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'111, mlcod=17'107)", 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.846220+0000", 2024-09-14T22:53:09.509 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 15.584886556000001, 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003845677, 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846220+0000", 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846220+0000", 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846220+0000", 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6100000000000001e-07 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846224+0000", 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.827e-06 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846231+0000", 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.063e-06 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846234+0000", 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4659999999999999e-06 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846263+0000", 2024-09-14T22:53:09.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8802999999999998e-05 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846276+0000", 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2714999999999999e-05 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850038+0000", 2024-09-14T22:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0037620549999999998 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850065+0000", 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7186999999999999e-05 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:158 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'107)", 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.846753+0000", 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 15.584353407, 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0033415490000000001, 2024-09-14T22:53:09.514 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846753+0000", 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846753+0000", 2024-09-14T22:53:09.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846753+0000", 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846820+0000", 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7288999999999999e-05 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846827+0000", 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3640000000000004e-06 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846830+0000", 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1e-06 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846856+0000", 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999999e-05 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846867+0000", 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1429e-05 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850066+0000", 2024-09-14T22:53:09.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031991089999999999 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850094+0000", 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7898e-05 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:159 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'113, mlcod=17'107)", 2024-09-14T22:53:09.519 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.847344+0000", 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 15.583762232, 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0029449179999999999, 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847344+0000", 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847344+0000", 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847344+0000", 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847348+0000", 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7720000000000001e-06 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847356+0000", 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847359+0000", 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1360000000000001e-06 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847395+0000", 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6647999999999999e-05 2024-09-14T22:53:09.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847408+0000", 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2130000000000001e-05 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850265+0000", 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0028577810000000002 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850289+0000", 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3405e-05 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.774468+0000", 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.65663826599999997, 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032723750000000001, 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.524 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 11 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774468+0000", 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774468+0000", 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774470+0000", 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7460000000000001e-06 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774478+0000", 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4570000000000005e-06 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774493+0000", 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5347999999999999e-05 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774502+0000", 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2910000000000007e-06 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774568+0000", 2024-09-14T22:53:09.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6527999999999997e-05 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774573+0000", 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.776325+0000", 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017522810000000001 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.528 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.776464+0000", 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000139326 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 5", 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.776684+0000", 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021937399999999999 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.777590+0000", 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00090645999999999999 2024-09-14T22:53:09.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.777679+0000", 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8632000000000001e-05 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.777710+0000", 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0639000000000003e-05 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.777740+0000", 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0642999999999998e-05 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.530 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.531 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.531 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.531 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.531 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:09.532 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:09.533 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:09.530+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:09.633 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:09.631+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:09.637 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:09.637 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:09.637 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:09.637 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:09.646 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:09.734 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:09.731+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:09.769 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:53:09.792 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:09.793 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:09.793 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:09.793 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:09.793 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:09.802 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:09.835 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:09.833+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:09.936 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:09.933+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:09.947 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4232.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:58.530337+0000", 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 11.416511352000001, 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00088297699999999996, 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.948 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530337+0000", 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530337+0000", 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530337+0000", 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.99e-07 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530344+0000", 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1650000000000003e-06 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530366+0000", 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2651999999999999e-05 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530379+0000", 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2789000000000001e-05 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530446+0000", 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6827000000000002e-05 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530473+0000", 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6874000000000001e-05 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.531172+0000", 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069956900000000004 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.531220+0000", 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7902000000000003e-05 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4232.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:59.536346+0000", 2024-09-14T22:53:09.952 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.410503013, 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010693110000000001, 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536346+0000", 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536346+0000", 2024-09-14T22:53:09.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536346+0000", 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8200000000000001e-07 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536361+0000", 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5047e-05 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536390+0000", 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8568000000000002e-05 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536403+0000", 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3142999999999999e-05 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536469+0000", 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5920999999999994e-05 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536497+0000", 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8124e-05 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.537378+0000", 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00088145900000000002 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.537415+0000", 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6667e-05 2024-09-14T22:53:09.956 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:9 3.1a e27/24 3:5cea7035:::rbd_info:head v 27'2, mlcod=0'0)", 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.764994+0000", 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.1818540479999999, 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000716671, 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.957 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.764994+0000", 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.764994+0000", 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.764995+0000", 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.92e-07 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765000+0000", 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3990000000000003e-06 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765008+0000", 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0960000000000001e-06 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765016+0000", 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0039999999999999e-06 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765059+0000", 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2378e-05 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765075+0000", 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6375000000000002e-05 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765679+0000", 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00060367199999999998 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765711+0000", 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2555000000000003e-05 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.961 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:12 3.8 e27/24 3:15b9a59a:::rbd_header.11352933567d:head v 27'1, mlcod=0'0)", 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.778991+0000", 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.167857594, 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00059963400000000004, 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778991+0000", 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778991+0000", 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778991+0000", 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.96e-07 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778997+0000", 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3349999999999997e-06 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779014+0000", 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6954000000000001e-05 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779021+0000", 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7249999999999997e-06 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779075+0000", 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3798000000000002e-05 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779089+0000", 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4119e-05 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779557+0000", 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046819599999999999 2024-09-14T22:53:09.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779591+0000", 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3210999999999998e-05 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:13 3.6 e27/24 3:62ea7de9:::rbd_object_map.11352933567d:head v 27'1, mlcod=0'0)", 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.780224+0000", 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.166624592, 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069506199999999998, 2024-09-14T22:53:09.966 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780224+0000", 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780224+0000", 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780224+0000", 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7000000000000001e-07 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780229+0000", 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780245+0000", 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5610000000000001e-05 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780251+0000", 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780305+0000", 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-05 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780324+0000", 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9845000000000001e-05 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780891+0000", 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654399999999999 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780919+0000", 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8031e-05 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.970 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:25 3.8 e27/24 3:15b9a59a:::rbd_header.11352933567d:head v 27'2, mlcod=0'0)", 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.788529+0000", 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.15831993, 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00072540699999999996, 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788529+0000", 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788529+0000", 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788530+0000", 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.517e-06 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788542+0000", 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1609e-05 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788561+0000", 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9051000000000001e-05 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788571+0000", 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9389999999999992e-06 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788640+0000", 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9671000000000004e-05 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788663+0000", 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2589e-05 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.789217+0000", 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00055367399999999998 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.789254+0000", 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7357000000000002e-05 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.974 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.975 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:09.975 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:09.975 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:09.976 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:10.037 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:10.035+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:10.114 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:10.122 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:10.122 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:10.122 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:10.122 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:10.132 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:10.138 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:10.136+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:10.238 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:10.239 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:10.239 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:10.236+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:10.255 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:10.255 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:10.271 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:10.271 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:10.279 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:10.279 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:10.288 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:10.289 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:10.289 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:10.305 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:10.305 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:10.322 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:10.322 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:10.339 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:10.337+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:10.340 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:10.340 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:10.356 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:10.357 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:10.411 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:10.411 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:10.424 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:10.467 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:10.465+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:10.502 INFO:teuthology.orchestra.run.smithi076.stdout:1 write 0xaaf4bd4 thru 0xaafe5b0 (0x99dd bytes) 2024-09-14T22:53:10.506 INFO:teuthology.orchestra.run.smithi076.stdout:2 write 0x60c78fb thru 0x60d5c1a (0xe320 bytes) 2024-09-14T22:53:10.519 INFO:teuthology.orchestra.run.smithi076.stdout:3 write 0xbb33c0d thru 0xbb43590 (0xf984 bytes) 2024-09-14T22:53:10.523 INFO:teuthology.orchestra.run.smithi076.stdout:4 punch from 0x3821ec0 to 0x3822144, (0x284 bytes) 2024-09-14T22:53:10.523 INFO:teuthology.orchestra.run.smithi076.stdout:5 write 0x650b070 thru 0x6519825 (0xe7b6 bytes) 2024-09-14T22:53:10.527 INFO:teuthology.orchestra.run.smithi076.stdout:6 write 0x1af9212 thru 0x1b0651d (0xd30c bytes) 2024-09-14T22:53:10.531 INFO:teuthology.orchestra.run.smithi076.stdout:truncating to largest ever: 0xa8593e9 2024-09-14T22:53:10.532 INFO:teuthology.orchestra.run.smithi076.stdout:7 trunc from 0xbb43591 to 0xa8593e9 2024-09-14T22:53:10.541 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:10.540+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:10.543 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:10.543 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:10.545 INFO:teuthology.orchestra.run.smithi076.stdout:8 writesame 0xa0561ac thru 0xa056e70 data_size 0x1(0xcc5 bytes) 2024-09-14T22:53:10.550 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.551 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.553 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:10.557 INFO:teuthology.orchestra.run.smithi076.stdout:12 trunc from 0xa8593e9 to 0x5757532 2024-09-14T22:53:10.571 INFO:teuthology.orchestra.run.smithi076.stdout:13 punch from 0x3869dc5 to 0x38780b6, (0xe2f1 bytes) 2024-09-14T22:53:10.584 INFO:teuthology.orchestra.run.smithi076.stdout:14 writesame 0x98f6286 thru 0x98fd805 data_size 0x80(0x7580 bytes) 2024-09-14T22:53:10.584 INFO:teuthology.orchestra.run.smithi076.stdout:16 read 0x4e41e02 thru 0x4e4c845 (0xaa44 bytes) 2024-09-14T22:53:10.584 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.584 INFO:teuthology.orchestra.run.smithi076.stdout:18 punch from 0xea56a8 to 0xeb4898, (0xf1f0 bytes) 2024-09-14T22:53:10.584 INFO:teuthology.orchestra.run.smithi076.stdout:19 write 0xccf187 thru 0xcdcd7c (0xdbf6 bytes) 2024-09-14T22:53:10.587 INFO:teuthology.orchestra.run.smithi076.stdout:20 write 0x5974e31 thru 0x597e5cc (0x979c bytes) 2024-09-14T22:53:10.591 INFO:teuthology.orchestra.run.smithi076.stdout:22 read 0x98449dc thru 0x984874c (0x3d71 bytes) 2024-09-14T22:53:10.591 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.591 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.591 INFO:teuthology.orchestra.run.smithi076.stdout:25 read 0x724359d thru 0x724faa5 (0xc509 bytes) 2024-09-14T22:53:10.591 INFO:teuthology.orchestra.run.smithi076.stdout:26 read 0x6e4b265 thru 0x6e5a70a (0xf4a6 bytes) 2024-09-14T22:53:10.591 INFO:teuthology.orchestra.run.smithi076.stdout:27 read 0x8806ef6 thru 0x880b46c (0x4577 bytes) 2024-09-14T22:53:10.591 INFO:teuthology.orchestra.run.smithi076.stdout:29 punch from 0x4e38bc4 to 0x4e44a87, (0xbec3 bytes) 2024-09-14T22:53:10.591 INFO:teuthology.orchestra.run.smithi076.stdout:30 write 0x6010ba4 thru 0x6016748 (0x5ba5 bytes) 2024-09-14T22:53:10.594 INFO:teuthology.orchestra.run.smithi076.stdout:31 read 0x19e02c2 thru 0x19e2992 (0x26d1 bytes) 2024-09-14T22:53:10.595 INFO:teuthology.orchestra.run.smithi076.stdout:32 read 0x797bce0 thru 0x79846a0 (0x89c1 bytes) 2024-09-14T22:53:10.595 INFO:teuthology.orchestra.run.smithi076.stdout:33 read 0x566bf05 thru 0x567b16e (0xf26a bytes) 2024-09-14T22:53:10.596 INFO:teuthology.orchestra.run.smithi076.stdout:34 punch from 0x298c62f to 0x299c27e, (0xfc4f bytes) 2024-09-14T22:53:10.596 INFO:teuthology.orchestra.run.smithi076.stdout:35 read 0x7959976 thru 0x7962a56 (0x90e1 bytes) 2024-09-14T22:53:10.596 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.596 INFO:teuthology.orchestra.run.smithi076.stdout:38 write 0x65bcf36 thru 0x65c6842 (0x990d bytes) 2024-09-14T22:53:10.602 INFO:teuthology.orchestra.run.smithi076.stdout:39 trunc from 0x98fd806 to 0x64b7585 2024-09-14T22:53:10.609 INFO:teuthology.orchestra.run.smithi076.stdout:truncating to largest ever: 0xe7ecc70 2024-09-14T22:53:10.619 INFO:teuthology.orchestra.run.smithi076.stdout:40 trunc from 0x64b7585 to 0xe7ecc70 2024-09-14T22:53:10.624 INFO:teuthology.orchestra.run.smithi076.stdout:41 read 0x592dacb thru 0x593940f (0xb945 bytes) 2024-09-14T22:53:10.625 INFO:teuthology.orchestra.run.smithi076.stdout:43 read 0x2edd543 thru 0x2eea1bb (0xcc79 bytes) 2024-09-14T22:53:10.625 INFO:teuthology.orchestra.run.smithi076.stdout:46 write 0x6a6a94a thru 0x6a75be5 (0xb29c bytes) 2024-09-14T22:53:10.628 INFO:teuthology.orchestra.run.smithi076.stdout:47 punch from 0x3c4cf14 to 0x3c5b91e, (0xea0a bytes) 2024-09-14T22:53:10.629 INFO:teuthology.orchestra.run.smithi076.stdout:48 write 0x2000672 thru 0x2008628 (0x7fb7 bytes) 2024-09-14T22:53:10.632 INFO:teuthology.orchestra.run.smithi076.stdout:51 write 0x8520337 thru 0x8525109 (0x4dd3 bytes) 2024-09-14T22:53:10.635 INFO:teuthology.orchestra.run.smithi076.stdout:52 read 0x1edcb8a thru 0x1ee776c (0xabe3 bytes) 2024-09-14T22:53:10.635 INFO:teuthology.orchestra.run.smithi076.stdout:53 write 0xa49fe2e thru 0xa4a1f44 (0x2117 bytes) 2024-09-14T22:53:10.638 INFO:teuthology.orchestra.run.smithi076.stdout:54 write 0x2bf6a3a thru 0x2bf6ab8 (0x7f bytes) 2024-09-14T22:53:10.641 INFO:teuthology.orchestra.run.smithi076.stdout:56 read 0x77171a0 thru 0x771a877 (0x36d8 bytes) 2024-09-14T22:53:10.641 INFO:teuthology.orchestra.run.smithi076.stdout:57 read 0x882b776 thru 0x8830ba8 (0x5433 bytes) 2024-09-14T22:53:10.642 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:10.641+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:10.644 INFO:teuthology.orchestra.run.smithi076.stdout:58 trunc from 0xe7ecc70 to 0xc643a2f 2024-09-14T22:53:10.649 INFO:teuthology.orchestra.run.smithi076.stdout:59 write 0x8917e9b thru 0x891fbca (0x7d30 bytes) 2024-09-14T22:53:10.652 INFO:teuthology.orchestra.run.smithi076.stdout:60 punch from 0xbf1948f to 0xbf19d79, (0x8ea bytes) 2024-09-14T22:53:10.652 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.652 INFO:teuthology.orchestra.run.smithi076.stdout:62 write 0x49b6ef4 thru 0x49c0ded (0x9efa bytes) 2024-09-14T22:53:10.661 INFO:teuthology.orchestra.run.smithi076.stdout:63 write 0xd19c4a9 thru 0xd1a5285 (0x8ddd bytes) 2024-09-14T22:53:10.664 INFO:teuthology.orchestra.run.smithi076.stdout:64 read 0x1dcddfa thru 0x1dcde06 (0xd bytes) 2024-09-14T22:53:10.664 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.665 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.665 INFO:teuthology.orchestra.run.smithi076.stdout:69 read 0x6f51bc9 thru 0x6f61728 (0xfb60 bytes) 2024-09-14T22:53:10.665 INFO:teuthology.orchestra.run.smithi076.stdout:70 write 0xbf02956 thru 0xbf0f744 (0xcdef bytes) 2024-09-14T22:53:10.667 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.667 INFO:teuthology.orchestra.run.smithi076.stdout:73 punch from 0x161a4b to 0x16c0d7, (0xa68c bytes) 2024-09-14T22:53:10.667 INFO:teuthology.orchestra.run.smithi076.stdout:74 read 0xadba9b6 thru 0xadc5e14 (0xb45f bytes) 2024-09-14T22:53:10.667 INFO:teuthology.orchestra.run.smithi076.stdout:76 write 0x4463bf1 thru 0x44694f6 (0x5906 bytes) 2024-09-14T22:53:10.670 INFO:teuthology.orchestra.run.smithi076.stdout:78 writesame 0xad13ae7 thru 0xad19a71 data_size 0x1f(0x5f8b bytes) 2024-09-14T22:53:10.680 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:10.680 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:10.680 INFO:teuthology.orchestra.run.smithi076.stdout:80 trunc from 0xd1a5286 to 0x74b170d 2024-09-14T22:53:10.690 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:10.691 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.698 INFO:teuthology.orchestra.run.smithi076.stdout:82 write 0x88c79b9 thru 0x88d3519 (0xbb61 bytes) 2024-09-14T22:53:10.711 INFO:teuthology.orchestra.run.smithi076.stdout:83 writesame 0xbd866c9 thru 0xbd873c3 data_size 0x1(0xcfb bytes) 2024-09-14T22:53:10.711 INFO:teuthology.orchestra.run.smithi076.stdout:84 read 0x68c0bad thru 0x68cd1e4 (0xc638 bytes) 2024-09-14T22:53:10.711 INFO:teuthology.orchestra.run.smithi076.stdout:85 punch from 0xaf9e70d to 0xafa8bc9, (0xa4bc bytes) 2024-09-14T22:53:10.719 INFO:teuthology.orchestra.run.smithi076.stdout:86 trunc from 0xbd873c4 to 0x4eded3f 2024-09-14T22:53:10.729 INFO:teuthology.orchestra.run.smithi076.stdout:87 punch from 0x2707182 to 0x270d9dc, (0x685a bytes) 2024-09-14T22:53:10.729 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.729 INFO:teuthology.orchestra.run.smithi076.stdout:89 read 0x17304b7 thru 0x173db64 (0xd6ae bytes) 2024-09-14T22:53:10.729 INFO:teuthology.orchestra.run.smithi076.stdout:90 punch from 0x45bd097 to 0x45c1578, (0x44e1 bytes) 2024-09-14T22:53:10.734 INFO:teuthology.orchestra.run.smithi076.stdout:91 write 0x53660f3 thru 0x536fc72 (0x9b80 bytes) 2024-09-14T22:53:10.737 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.738 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.738 INFO:teuthology.orchestra.run.smithi076.stdout:96 punch from 0x2d45ec9 to 0x2d4bad5, (0x5c0c bytes) 2024-09-14T22:53:10.742 INFO:teuthology.orchestra.run.smithi076.stdout:97 write 0x5570973 thru 0x5576fdd (0x666b bytes) 2024-09-14T22:53:10.743 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:10.741+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:10.745 INFO:teuthology.orchestra.run.smithi076.stdout:99 read 0x837c03 thru 0x844e08 (0xd206 bytes) 2024-09-14T22:53:10.745 INFO:teuthology.orchestra.run.smithi076.stdout:101 writesame 0xf8acab thru 0xf8ccf9 data_size 0x3(0x204f bytes) 2024-09-14T22:53:10.763 INFO:teuthology.orchestra.run.smithi076.stdout:106 writesame 0xd5c5021 thru 0xd5c79ff data_size 0x9(0x29df bytes) 2024-09-14T22:53:10.764 INFO:teuthology.orchestra.run.smithi076.stdout:107 write 0x853004e thru 0x853b8fe (0xb8b1 bytes) 2024-09-14T22:53:10.767 INFO:teuthology.orchestra.run.smithi076.stdout:109 punch from 0x493a23c to 0x493e699, (0x445d bytes) 2024-09-14T22:53:10.767 INFO:teuthology.orchestra.run.smithi076.stdout:112 writesame 0x4f2cd91 thru 0x4f35cd3 data_size 0x4b(0x8f43 bytes) 2024-09-14T22:53:10.807 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:10.807 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:10.816 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:10.856 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:10.854+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:10.921 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:10.921 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:10.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:10.969 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.969 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.969 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.972 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:10.971+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:10.977 INFO:teuthology.orchestra.run.smithi076.stdout:117 trunc from 0xd5c7a00 to 0x632b919 2024-09-14T22:53:10.987 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:10.987 INFO:teuthology.orchestra.run.smithi076.stdout:120 read 0x3cfeb1d thru 0x3d05ada (0x6fbe bytes) 2024-09-14T22:53:10.994 INFO:teuthology.orchestra.run.smithi076.stdout:121 write 0x6ab94c5 thru 0x6ac6421 (0xcf5d bytes) 2024-09-14T22:53:10.997 INFO:teuthology.orchestra.run.smithi076.stdout:122 read 0x51f7277 thru 0x520482e (0xd5b8 bytes) 2024-09-14T22:53:10.998 INFO:teuthology.orchestra.run.smithi076.stdout:123 punch from 0x3af453f to 0x3afeef0, (0xa9b1 bytes) 2024-09-14T22:53:10.998 INFO:teuthology.orchestra.run.smithi076.stdout:124 read 0x5e08003 thru 0x5e0c8ae (0x48ac bytes) 2024-09-14T22:53:10.998 INFO:teuthology.orchestra.run.smithi076.stdout:125 trunc from 0x6ac6422 to 0x666cb89 2024-09-14T22:53:11.006 INFO:teuthology.orchestra.run.smithi076.stdout:126 read 0x3cda305 thru 0x3cda58d (0x289 bytes) 2024-09-14T22:53:11.006 INFO:teuthology.orchestra.run.smithi076.stdout:127 write 0x1cfc64c thru 0x1cfea06 (0x23bb bytes) 2024-09-14T22:53:11.010 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.019 INFO:teuthology.orchestra.run.smithi076.stdout:130 writesame 0xa2c91a6 thru 0xa2ca385 data_size 0x2c(0x11e0 bytes) 2024-09-14T22:53:11.019 INFO:teuthology.orchestra.run.smithi076.stdout:131 read 0x46392fa thru 0x46424d5 (0x91dc bytes) 2024-09-14T22:53:11.031 INFO:teuthology.orchestra.run.smithi076.stdout:132 trunc from 0xa2ca386 to 0x3b0cc3 2024-09-14T22:53:11.034 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:11.034 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:11.044 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:11.045 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:11.042+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:11.053 INFO:teuthology.orchestra.run.smithi076.stdout:134 writesame 0xe0d1c0 thru 0xe19768 data_size 0x65(0xc5a9 bytes) 2024-09-14T22:53:11.061 INFO:teuthology.orchestra.run.smithi076.stdout:135 write 0x1f9739a thru 0x1fa10e3 (0x9d4a bytes) 2024-09-14T22:53:11.082 INFO:teuthology.orchestra.run.smithi076.stdout:137 writesame 0xcdc246d thru 0xcdc4a7b data_size 0x1(0x260f bytes) 2024-09-14T22:53:11.083 INFO:teuthology.orchestra.run.smithi076.stdout:138 write 0x9e207a7 thru 0x9e262d8 (0x5b32 bytes) 2024-09-14T22:53:11.088 INFO:teuthology.orchestra.run.smithi076.stdout:139 writesame 0x3168d6e thru 0x3170e49 data_size 0x6(0x80dc bytes) 2024-09-14T22:53:11.093 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.101 INFO:teuthology.orchestra.run.smithi076.stdout:142 writesame 0xdb7e476 thru 0xdb7ee81 data_size 0x2(0xa0c bytes) 2024-09-14T22:53:11.101 INFO:teuthology.orchestra.run.smithi076.stdout:143 write 0x7572bfe thru 0x7579c92 (0x7095 bytes) 2024-09-14T22:53:11.104 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.104 INFO:teuthology.orchestra.run.smithi076.stdout:145 punch from 0x8731ebe to 0x873b88c, (0x99ce bytes) 2024-09-14T22:53:11.104 INFO:teuthology.orchestra.run.smithi076.stdout:146 read 0xa0f1b4b thru 0xa0f3546 (0x19fc bytes) 2024-09-14T22:53:11.104 INFO:teuthology.orchestra.run.smithi076.stdout:147 writesame 0x2bf1296 thru 0x2bf888d data_size 0x19(0x75f8 bytes) 2024-09-14T22:53:11.107 INFO:teuthology.orchestra.run.smithi076.stdout:148 writesame 0xa8e296c thru 0xa8ea014 data_size 0x1(0x76a9 bytes) 2024-09-14T22:53:11.127 INFO:teuthology.orchestra.run.smithi076.stdout:149 read 0x990206 thru 0x99e875 (0xe670 bytes) 2024-09-14T22:53:11.127 INFO:teuthology.orchestra.run.smithi076.stdout:151 write 0xbb132c0 thru 0xbb174d9 (0x421a bytes) 2024-09-14T22:53:11.131 INFO:teuthology.orchestra.run.smithi076.stdout:152 read 0xcfa08a7 thru 0xcfa3e0f (0x3569 bytes) 2024-09-14T22:53:11.131 INFO:teuthology.orchestra.run.smithi076.stdout:153 read 0x8916dc6 thru 0x892558f (0xe7ca bytes) 2024-09-14T22:53:11.132 INFO:teuthology.orchestra.run.smithi076.stdout:155 write 0xd2359f5 thru 0xd23f0a8 (0x96b4 bytes) 2024-09-14T22:53:11.135 INFO:teuthology.orchestra.run.smithi076.stdout:156 write 0xd7424c8 thru 0xd742f0c (0xa45 bytes) 2024-09-14T22:53:11.138 INFO:teuthology.orchestra.run.smithi076.stdout:157 punch from 0x4c8d19d to 0x4c8efdd, (0x1e40 bytes) 2024-09-14T22:53:11.146 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:11.144+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:11.147 INFO:teuthology.orchestra.run.smithi076.stdout:158 trunc from 0xdb7ee82 to 0x60b3be1 2024-09-14T22:53:11.163 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.163 INFO:teuthology.orchestra.run.smithi076.stdout:160 read 0x1ab520 thru 0x1b13dd (0x5ebe bytes) 2024-09-14T22:53:11.163 INFO:teuthology.orchestra.run.smithi076.stdout:161 punch from 0x54feeb to 0x55b39f, (0xb4b4 bytes) 2024-09-14T22:53:11.163 INFO:teuthology.orchestra.run.smithi076.stdout:162 punch from 0x221a00b to 0x22282a2, (0xe297 bytes) 2024-09-14T22:53:11.175 INFO:teuthology.orchestra.run.smithi076.stdout:163 write 0xc01363c thru 0xc018895 (0x525a bytes) 2024-09-14T22:53:11.178 INFO:teuthology.orchestra.run.smithi076.stdout:164 write 0xadd8056 thru 0xade384e (0xb7f9 bytes) 2024-09-14T22:53:11.181 INFO:teuthology.orchestra.run.smithi076.stdout:165 read 0x9984814 thru 0x998baf9 (0x72e6 bytes) 2024-09-14T22:53:11.181 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.181 INFO:teuthology.orchestra.run.smithi076.stdout:167 writesame 0xbee9a4 thru 0xbfb3e7 data_size 0x1e(0xca44 bytes) 2024-09-14T22:53:11.185 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.185 INFO:teuthology.orchestra.run.smithi076.stdout:169 read 0x676ad6c thru 0x67792e5 (0xe57a bytes) 2024-09-14T22:53:11.185 INFO:teuthology.orchestra.run.smithi076.stdout:171 write 0x2983195 thru 0x2989fdd (0x6e49 bytes) 2024-09-14T22:53:11.188 INFO:teuthology.orchestra.run.smithi076.stdout:172 read 0x52cc24d thru 0x52dbc5b (0xfa0f bytes) 2024-09-14T22:53:11.188 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.188 INFO:teuthology.orchestra.run.smithi076.stdout:174 write 0x1582fa2 thru 0x15866e1 (0x3740 bytes) 2024-09-14T22:53:11.190 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:11.190 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:11.196 INFO:teuthology.orchestra.run.smithi076.stdout:175 writesame 0xc58410c thru 0xc58947f data_size 0xe(0x5374 bytes) 2024-09-14T22:53:11.196 INFO:teuthology.orchestra.run.smithi076.stdout:176 read 0x6430322 thru 0x6436606 (0x62e5 bytes) 2024-09-14T22:53:11.196 INFO:teuthology.orchestra.run.smithi076.stdout:177 read 0x722f49f thru 0x7238c6c (0x97ce bytes) 2024-09-14T22:53:11.197 INFO:teuthology.orchestra.run.smithi076.stdout:179 read 0xb0f8f77 thru 0xb104d63 (0xbded bytes) 2024-09-14T22:53:11.199 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:11.202 INFO:teuthology.orchestra.run.smithi076.stdout:182 trunc from 0xc589480 to 0x7e1f14f 2024-09-14T22:53:11.212 INFO:teuthology.orchestra.run.smithi076.stdout:184 write 0x4d5c637 thru 0x4d5e47e (0x1e48 bytes) 2024-09-14T22:53:11.215 INFO:teuthology.orchestra.run.smithi076.stdout:185 punch from 0xfc48fa to 0xfcf2f7, (0xa9fd bytes) 2024-09-14T22:53:11.215 INFO:teuthology.orchestra.run.smithi076.stdout:186 write 0x4d78888 thru 0x4d85824 (0xcf9d bytes) 2024-09-14T22:53:11.225 INFO:teuthology.orchestra.run.smithi076.stdout:187 write 0x9fcac7b thru 0x9fd7ee2 (0xd268 bytes) 2024-09-14T22:53:11.229 INFO:teuthology.orchestra.run.smithi076.stdout:188 read 0x49e0a56 thru 0x49e22d5 (0x1880 bytes) 2024-09-14T22:53:11.229 INFO:teuthology.orchestra.run.smithi076.stdout:189 write 0x34a305c thru 0x34a9afb (0x6aa0 bytes) 2024-09-14T22:53:11.232 INFO:teuthology.orchestra.run.smithi076.stdout:190 write 0x916508 thru 0x91d9c2 (0x74bb bytes) 2024-09-14T22:53:11.233 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.234 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.234 INFO:teuthology.orchestra.run.smithi076.stdout:194 read 0x6629a7a thru 0x6632185 (0x870c bytes) 2024-09-14T22:53:11.234 INFO:teuthology.orchestra.run.smithi076.stdout:195 punch from 0x98efaa to 0x992263, (0x32b9 bytes) 2024-09-14T22:53:11.234 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.238 INFO:teuthology.orchestra.run.smithi076.stdout:197 writesame 0xa311276 thru 0xa31bf42 data_size 0x1(0xaccd bytes) 2024-09-14T22:53:11.238 INFO:teuthology.orchestra.run.smithi076.stdout:198 write 0x307edc5 thru 0x3084855 (0x5a91 bytes) 2024-09-14T22:53:11.240 INFO:teuthology.orchestra.run.smithi076.stdout:201 punch from 0x3c66427 to 0x3c6cb1a, (0x66f3 bytes) 2024-09-14T22:53:11.241 INFO:teuthology.orchestra.run.smithi076.stdout:202 trunc from 0xa31bf43 to 0xaf6dd11 2024-09-14T22:53:11.247 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:11.245+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:11.255 INFO:teuthology.orchestra.run.smithi076.stdout:203 writesame 0xece5483 thru 0xecf2678 data_size 0x5(0xd1f6 bytes) 2024-09-14T22:53:11.255 INFO:teuthology.orchestra.run.smithi076.stdout:204 write 0x320fef8 thru 0x3219201 (0x930a bytes) 2024-09-14T22:53:11.257 INFO:teuthology.orchestra.run.smithi076.stdout:207 read 0x15f6925 thru 0x15fb01c (0x46f8 bytes) 2024-09-14T22:53:11.258 INFO:teuthology.orchestra.run.smithi076.stdout:210 read 0xadf08ec thru 0xadfa3dc (0x9af1 bytes) 2024-09-14T22:53:11.258 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.258 INFO:teuthology.orchestra.run.smithi076.stdout:214 write 0x258ff30 thru 0x2599889 (0x995a bytes) 2024-09-14T22:53:11.274 INFO:teuthology.orchestra.run.smithi076.stdout:217 trunc from 0xecf2679 to 0x2d9b5c2 2024-09-14T22:53:11.304 INFO:teuthology.orchestra.run.smithi076.stdout:218 write 0xcb9d55a thru 0xcba69d1 (0x9478 bytes) 2024-09-14T22:53:11.305 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:11.305 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:11.307 INFO:teuthology.orchestra.run.smithi076.stdout:219 read 0x982127a thru 0x982bc9f (0xaa26 bytes) 2024-09-14T22:53:11.307 INFO:teuthology.orchestra.run.smithi076.stdout:220 punch from 0x5b8c854 to 0x5b8fbe1, (0x338d bytes) 2024-09-14T22:53:11.307 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.307 INFO:teuthology.orchestra.run.smithi076.stdout:224 read 0xc713748 thru 0xc718ea0 (0x5759 bytes) 2024-09-14T22:53:11.307 INFO:teuthology.orchestra.run.smithi076.stdout:225 read 0x704d8d7 thru 0x7052567 (0x4c91 bytes) 2024-09-14T22:53:11.307 INFO:teuthology.orchestra.run.smithi076.stdout:226 writesame 0xb0fe67c thru 0xb106b30 data_size 0x1(0x84b5 bytes) 2024-09-14T22:53:11.314 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:11.328 INFO:teuthology.orchestra.run.smithi076.stdout:227 read 0x86134b3 thru 0x86223e5 (0xef33 bytes) 2024-09-14T22:53:11.328 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.328 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.328 INFO:teuthology.orchestra.run.smithi076.stdout:232 read 0x9a4f64a thru 0x9a5f61d (0xffd4 bytes) 2024-09-14T22:53:11.334 INFO:teuthology.orchestra.run.smithi076.stdout:233 trunc from 0xcba69d2 to 0x783df4e 2024-09-14T22:53:11.346 INFO:teuthology.orchestra.run.smithi076.stdout:235 read 0x4721889 thru 0x4729718 (0x7e90 bytes) 2024-09-14T22:53:11.357 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:11.356+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:11.358 INFO:teuthology.orchestra.run.smithi076.stdout:236 writesame 0xdced33a thru 0xdcfb477 data_size 0xb(0xe13e bytes) 2024-09-14T22:53:11.358 INFO:teuthology.orchestra.run.smithi076.stdout:237 writesame 0x9eb8af1 thru 0x9ec5da6 data_size 0x7(0xd2b6 bytes) 2024-09-14T22:53:11.364 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.365 INFO:teuthology.orchestra.run.smithi076.stdout:239 punch from 0x1e0a4eb to 0x1e0f5ed, (0x5102 bytes) 2024-09-14T22:53:11.365 INFO:teuthology.orchestra.run.smithi076.stdout:240 writesame 0xd08dba3 thru 0xd097a3b data_size 0x1(0x9e99 bytes) 2024-09-14T22:53:11.388 INFO:teuthology.orchestra.run.smithi076.stdout:241 writesame 0x9558eaa thru 0x955e6b8 data_size 0x1(0x580f bytes) 2024-09-14T22:53:11.405 INFO:teuthology.orchestra.run.smithi076.stdout:242 trunc from 0xdcfb478 to 0xa6febfb 2024-09-14T22:53:11.417 INFO:teuthology.orchestra.run.smithi076.stdout:243 punch from 0x97c7a0 to 0x981631, (0x4e91 bytes) 2024-09-14T22:53:11.417 INFO:teuthology.orchestra.run.smithi076.stdout:244 read 0x7b487ce thru 0x7b57e74 (0xf6a7 bytes) 2024-09-14T22:53:11.417 INFO:teuthology.orchestra.run.smithi076.stdout:246 writesame 0x4c94864 thru 0x4c960d1 data_size 0x2(0x186e bytes) 2024-09-14T22:53:11.418 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:11.419 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:11.421 INFO:teuthology.orchestra.run.smithi076.stdout:247 read 0x71bd4f6 thru 0x71c4608 (0x7113 bytes) 2024-09-14T22:53:11.421 INFO:teuthology.orchestra.run.smithi076.stdout:248 punch from 0x6d6495b to 0x6d6e462, (0x9b07 bytes) 2024-09-14T22:53:11.421 INFO:teuthology.orchestra.run.smithi076.stdout:249 read 0x9386d47 thru 0x9396beb (0xfea5 bytes) 2024-09-14T22:53:11.421 INFO:teuthology.orchestra.run.smithi076.stdout:250 writesame 0x520ad13 thru 0x520eb8c data_size 0xb(0x3e7a bytes) 2024-09-14T22:53:11.424 INFO:teuthology.orchestra.run.smithi076.stdout:252 punch from 0x5f1b0e5 to 0x5f227dc, (0x76f7 bytes) 2024-09-14T22:53:11.424 INFO:teuthology.orchestra.run.smithi076.stdout:253 punch from 0x1b058e2 to 0x1b0c5a2, (0x6cc0 bytes) 2024-09-14T22:53:11.425 INFO:teuthology.orchestra.run.smithi076.stdout:254 write 0x8821c58 thru 0x8823976 (0x1d1f bytes) 2024-09-14T22:53:11.427 INFO:teuthology.orchestra.run.smithi076.stdout:256 write 0x9fa9b76 thru 0x9fb23dd (0x8868 bytes) 2024-09-14T22:53:11.428 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:11.430 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.438 INFO:teuthology.orchestra.run.smithi076.stdout:258 write 0xdbcb81c thru 0xdbda92e (0xf113 bytes) 2024-09-14T22:53:11.442 INFO:teuthology.orchestra.run.smithi076.stdout:259 read 0x3bb98d4 thru 0x3bc53c4 (0xbaf1 bytes) 2024-09-14T22:53:11.442 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.442 INFO:teuthology.orchestra.run.smithi076.stdout:262 punch from 0x601973a to 0x6028ecf, (0xf795 bytes) 2024-09-14T22:53:11.442 INFO:teuthology.orchestra.run.smithi076.stdout:265 write 0xa3f10f2 thru 0xa3fe630 (0xd53f bytes) 2024-09-14T22:53:11.448 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.448 INFO:teuthology.orchestra.run.smithi076.stdout:269 write 0x252261c thru 0x2532372 (0xfd57 bytes) 2024-09-14T22:53:11.449 INFO:teuthology.orchestra.run.smithi076.stdout:270 read 0x3410bfb thru 0x3415742 (0x4b48 bytes) 2024-09-14T22:53:11.449 INFO:teuthology.orchestra.run.smithi076.stdout:271 read 0xbfc8eb9 thru 0xbfccd76 (0x3ebe bytes) 2024-09-14T22:53:11.450 INFO:teuthology.orchestra.run.smithi076.stdout:272 read 0x6fff8d5 thru 0x7004cc4 (0x53f0 bytes) 2024-09-14T22:53:11.457 INFO:teuthology.orchestra.run.smithi076.stdout:274 trunc from 0xdbda92f to 0x7c7fa5d 2024-09-14T22:53:11.469 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:11.468+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:11.474 INFO:teuthology.orchestra.run.smithi076.stdout:275 trunc from 0x7c7fa5d to 0x6199815 2024-09-14T22:53:11.480 INFO:teuthology.orchestra.run.smithi076.stdout:276 punch from 0xf4e3b to 0xfad09, (0x5ece bytes) 2024-09-14T22:53:11.480 INFO:teuthology.orchestra.run.smithi076.stdout:277 read 0x3d973a3 thru 0x3da0c6c (0x98ca bytes) 2024-09-14T22:53:11.480 INFO:teuthology.orchestra.run.smithi076.stdout:truncating to largest ever: 0xebbf34b 2024-09-14T22:53:11.490 INFO:teuthology.orchestra.run.smithi076.stdout:278 trunc from 0x6199815 to 0xebbf34b 2024-09-14T22:53:11.495 INFO:teuthology.orchestra.run.smithi076.stdout:279 write 0x7e94233 thru 0x7e98889 (0x4657 bytes) 2024-09-14T22:53:11.499 INFO:teuthology.orchestra.run.smithi076.stdout:280 punch from 0xdee57da to 0xdee722b, (0x1a51 bytes) 2024-09-14T22:53:11.499 INFO:teuthology.orchestra.run.smithi076.stdout:282 punch from 0x2b29646 to 0x2b2a316, (0xcd0 bytes) 2024-09-14T22:53:11.499 INFO:teuthology.orchestra.run.smithi076.stdout:283 write 0x2f9c105 thru 0x2fa1daa (0x5ca6 bytes) 2024-09-14T22:53:11.501 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.513 INFO:teuthology.orchestra.run.smithi076.stdout:285 trunc from 0xebbf34b to 0x4a6fed1 2024-09-14T22:53:11.526 INFO:teuthology.orchestra.run.smithi076.stdout:287 punch from 0x45f1e4b to 0x45f7d81, (0x5f36 bytes) 2024-09-14T22:53:11.526 INFO:teuthology.orchestra.run.smithi076.stdout:288 read 0x17a7c1e thru 0x17af983 (0x7d66 bytes) 2024-09-14T22:53:11.527 INFO:teuthology.orchestra.run.smithi076.stdout:289 read 0x244eef2 thru 0x24535a9 (0x46b8 bytes) 2024-09-14T22:53:11.527 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.542 INFO:teuthology.orchestra.run.smithi076.stdout:291 write 0xb2c61a0 thru 0xb2d0636 (0xa497 bytes) 2024-09-14T22:53:11.546 INFO:teuthology.orchestra.run.smithi076.stdout:292 write 0x41eb25f thru 0x41f95b3 (0xe355 bytes) 2024-09-14T22:53:11.549 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:11.547+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:11.551 INFO:teuthology.orchestra.run.smithi076.stdout:297 write 0x5c98ecb thru 0x5ca2801 (0x9937 bytes) 2024-09-14T22:53:11.554 INFO:teuthology.orchestra.run.smithi076.stdout:298 read 0x7ad75ee thru 0x7ae29fc (0xb40f bytes) 2024-09-14T22:53:11.554 INFO:teuthology.orchestra.run.smithi076.stdout:300 write 0x6da0498 thru 0x6da5706 (0x526f bytes) 2024-09-14T22:53:11.556 INFO:teuthology.orchestra.run.smithi076.stdout:301 write 0x1cb78bc thru 0x1cb9742 (0x1e87 bytes) 2024-09-14T22:53:11.561 INFO:teuthology.orchestra.run.smithi076.stdout:302 punch from 0x9b36a5e to 0x9b3b0d4, (0x4676 bytes) 2024-09-14T22:53:11.561 INFO:teuthology.orchestra.run.smithi076.stdout:303 writesame 0x460536e thru 0x460b4f9 data_size 0x6(0x618c bytes) 2024-09-14T22:53:11.566 INFO:teuthology.orchestra.run.smithi076.stdout:304 punch from 0x8467952 to 0x846dcb9, (0x6367 bytes) 2024-09-14T22:53:11.566 INFO:teuthology.orchestra.run.smithi076.stdout:305 read 0x769fbea thru 0x76a9e41 (0xa258 bytes) 2024-09-14T22:53:11.569 INFO:teuthology.orchestra.run.smithi076.stdout:307 trunc from 0xb2d0637 to 0xdaf10f6 2024-09-14T22:53:11.574 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:11.574 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:11.574 INFO:teuthology.orchestra.run.smithi076.stdout:308 punch from 0x1421200 to 0x14253b4, (0x41b4 bytes) 2024-09-14T22:53:11.575 INFO:teuthology.orchestra.run.smithi076.stdout:309 write 0x6652da8 thru 0x6656d96 (0x3fef bytes) 2024-09-14T22:53:11.583 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:11.587 INFO:teuthology.orchestra.run.smithi076.stdout:310 trunc from 0xdaf10f6 to 0x5ad2828 2024-09-14T22:53:11.604 INFO:teuthology.orchestra.run.smithi076.stdout:311 trunc from 0x5ad2828 to 0xbd40e21 2024-09-14T22:53:11.609 INFO:teuthology.orchestra.run.smithi076.stdout:313 write 0x14a03a4 thru 0x14ac381 (0xbfde bytes) 2024-09-14T22:53:11.611 INFO:teuthology.orchestra.run.smithi076.stdout:318 writesame 0x77546ba thru 0x775ca37 data_size 0x1(0x837e bytes) 2024-09-14T22:53:11.628 INFO:teuthology.orchestra.run.smithi076.stdout:319 writesame 0x63bfe58 thru 0x63c7eb7 data_size 0x34(0x8060 bytes) 2024-09-14T22:53:11.633 INFO:teuthology.orchestra.run.smithi076.stdout:323 read 0x14073ec thru 0x140a5f2 (0x3207 bytes) 2024-09-14T22:53:11.633 INFO:teuthology.orchestra.run.smithi076.stdout:324 read 0x6fc6b05 thru 0x6fce3f4 (0x78f0 bytes) 2024-09-14T22:53:11.633 INFO:teuthology.orchestra.run.smithi076.stdout:325 punch from 0x3ce3be0 to 0x3ce9601, (0x5a21 bytes) 2024-09-14T22:53:11.634 INFO:teuthology.orchestra.run.smithi076.stdout:327 read 0x49ec3d5 thru 0x49f6166 (0x9d92 bytes) 2024-09-14T22:53:11.634 INFO:teuthology.orchestra.run.smithi076.stdout:328 writesame 0x5d7685 thru 0x5e563e data_size 0x2(0xdfba bytes) 2024-09-14T22:53:11.649 INFO:teuthology.orchestra.run.smithi076.stdout:330 read 0x22bd694 thru 0x22c112f (0x3a9c bytes) 2024-09-14T22:53:11.649 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:11.647+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:11.660 INFO:teuthology.orchestra.run.smithi076.stdout:331 trunc from 0xbd40e21 to 0x2a992d8 2024-09-14T22:53:11.674 INFO:teuthology.orchestra.run.smithi076.stdout:334 write 0x1da93f1 thru 0x1dacd05 (0x3915 bytes) 2024-09-14T22:53:11.675 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.683 INFO:teuthology.orchestra.run.smithi076.stdout:336 write 0x54cd693 thru 0x54d363d (0x5fab bytes) 2024-09-14T22:53:11.699 INFO:teuthology.orchestra.run.smithi076.stdout:338 write 0xd02234e thru 0xd0279c3 (0x5676 bytes) 2024-09-14T22:53:11.702 INFO:teuthology.orchestra.run.smithi076.stdout:339 write 0x23a2dc2 thru 0x23af37a (0xc5b9 bytes) 2024-09-14T22:53:11.706 INFO:teuthology.orchestra.run.smithi076.stdout:340 write 0x7d3646e thru 0x7d3a9c7 (0x455a bytes) 2024-09-14T22:53:11.709 INFO:teuthology.orchestra.run.smithi076.stdout:341 write 0x56ee0c3 thru 0x56f28ad (0x47eb bytes) 2024-09-14T22:53:11.711 INFO:teuthology.orchestra.run.smithi076.stdout:342 read 0x9ca878e thru 0x9ca9e78 (0x16eb bytes) 2024-09-14T22:53:11.714 INFO:teuthology.orchestra.run.smithi076.stdout:344 trunc from 0xd0279c4 to 0x9cf9668 2024-09-14T22:53:11.724 INFO:teuthology.orchestra.run.smithi076.stdout:345 read 0x22b70ab thru 0x22be009 (0x6f5f bytes) 2024-09-14T22:53:11.724 INFO:teuthology.orchestra.run.smithi076.stdout:346 read 0xbd9f24 thru 0xbe8738 (0xe815 bytes) 2024-09-14T22:53:11.730 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:11.730 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:11.735 INFO:teuthology.orchestra.run.smithi076.stdout:347 writesame 0xe0bbf8b thru 0xe0c508b data_size 0x1(0x9101 bytes) 2024-09-14T22:53:11.735 INFO:teuthology.orchestra.run.smithi076.stdout:348 punch from 0xffd9d9 to 0x1002317, (0x493e bytes) 2024-09-14T22:53:11.739 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:11.741 INFO:teuthology.orchestra.run.smithi076.stdout:350 trunc from 0xe0c508c to 0x92496ed 2024-09-14T22:53:11.746 INFO:teuthology.orchestra.run.smithi076.stdout:355 write 0xa68172 thru 0xa735e9 (0xb478 bytes) 2024-09-14T22:53:11.748 INFO:teuthology.orchestra.run.smithi076.stdout:356 punch from 0x29e8ec0 to 0x29f23ab, (0x94eb bytes) 2024-09-14T22:53:11.749 INFO:teuthology.orchestra.run.smithi076.stdout:357 punch from 0x53711d7 to 0x53746a4, (0x34cd bytes) 2024-09-14T22:53:11.749 INFO:teuthology.orchestra.run.smithi076.stdout:358 punch from 0x5fc76f6 to 0x5fc8522, (0xe2c bytes) 2024-09-14T22:53:11.749 INFO:teuthology.orchestra.run.smithi076.stdout:359 read 0x291846 thru 0x292a11 (0x11cc bytes) 2024-09-14T22:53:11.749 INFO:teuthology.orchestra.run.smithi076.stdout:360 punch from 0x8c98bc4 to 0x8c9d3cc, (0x4808 bytes) 2024-09-14T22:53:11.749 INFO:teuthology.orchestra.run.smithi076.stdout:361 punch from 0x8de1fa8 to 0x8de8fff, (0x7057 bytes) 2024-09-14T22:53:11.749 INFO:teuthology.orchestra.run.smithi076.stdout:362 read 0x517136c thru 0x5176ee8 (0x5b7d bytes) 2024-09-14T22:53:11.749 INFO:teuthology.orchestra.run.smithi076.stdout:364 read 0x7a727e3 thru 0x7a72800 (0x1e bytes) 2024-09-14T22:53:11.757 INFO:teuthology.orchestra.run.smithi076.stdout:365 writesame 0xc532b43 thru 0xc53b800 data_size 0xf(0x8cbe bytes) 2024-09-14T22:53:11.764 INFO:teuthology.orchestra.run.smithi076.stdout:366 write 0xdf76b2c thru 0xdf80bb0 (0xa085 bytes) 2024-09-14T22:53:11.767 INFO:teuthology.orchestra.run.smithi076.stdout:367 write 0xd020c51 thru 0xd02def3 (0xd2a3 bytes) 2024-09-14T22:53:11.770 INFO:teuthology.orchestra.run.smithi076.stdout:369 read 0x7ec6cf6 thru 0x7ed219e (0xb4a9 bytes) 2024-09-14T22:53:11.771 INFO:teuthology.orchestra.run.smithi076.stdout:370 read 0x3e1f8f2 thru 0x3e25752 (0x5e61 bytes) 2024-09-14T22:53:11.775 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:11.775 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:11.775 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:53:11.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:53:11.776 INFO:teuthology.orchestra.run.smithi076.stdout:371 write 0xe76a2a4 thru 0xe779867 (0xf5c4 bytes) 2024-09-14T22:53:11.780 INFO:teuthology.orchestra.run.smithi076.stdout:372 read 0x8a459e2 thru 0x8a4bda9 (0x63c8 bytes) 2024-09-14T22:53:11.780 INFO:teuthology.orchestra.run.smithi076.stdout:373 read 0x6ddbc9b thru 0x6de68e4 (0xac4a bytes) 2024-09-14T22:53:11.780 INFO:teuthology.orchestra.run.smithi076.stdout:374 read 0x4ceb58a thru 0x4cf0134 (0x4bab bytes) 2024-09-14T22:53:11.783 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:53:11.783 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:11.782+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:11.794 INFO:teuthology.orchestra.run.smithi076.stdout:375 trunc from 0xe779868 to 0x29c1b1a 2024-09-14T22:53:11.822 INFO:teuthology.orchestra.run.smithi076.stdout:376 trunc from 0x29c1b1a to 0xe4b3b9e 2024-09-14T22:53:11.828 INFO:teuthology.orchestra.run.smithi076.stdout:377 write 0xd4c5bd7 thru 0xd4d285a (0xcc84 bytes) 2024-09-14T22:53:11.832 INFO:teuthology.orchestra.run.smithi076.stdout:378 read 0xe0cd2a thru 0xe0e822 (0x1af9 bytes) 2024-09-14T22:53:11.844 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:11.844 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:11.845 INFO:teuthology.orchestra.run.smithi076.stdout:379 trunc from 0xe4b3b9e to 0x559c6a8 2024-09-14T22:53:11.851 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:11.849+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:11.853 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:11.858 INFO:teuthology.orchestra.run.smithi076.stdout:380 punch from 0x3f20cbd to 0x3f2a625, (0x9968 bytes) 2024-09-14T22:53:11.872 INFO:teuthology.orchestra.run.smithi076.stdout:381 write 0xc0be576 thru 0xc0c5422 (0x6ead bytes) 2024-09-14T22:53:11.875 INFO:teuthology.orchestra.run.smithi076.stdout:383 read 0xa679cc thru 0xa6a63c (0x2c71 bytes) 2024-09-14T22:53:11.876 INFO:teuthology.orchestra.run.smithi076.stdout:384 punch from 0xb7e32cb to 0xb7e9653, (0x6388 bytes) 2024-09-14T22:53:11.876 INFO:teuthology.orchestra.run.smithi076.stdout:386 write 0x9dcd2f0 thru 0x9dcff39 (0x2c4a bytes) 2024-09-14T22:53:11.893 INFO:teuthology.orchestra.run.smithi076.stdout:387 trunc from 0xc0c5423 to 0x5125ff 2024-09-14T22:53:11.907 INFO:teuthology.orchestra.run.smithi076.stdout:388 read 0x1b481e thru 0x1c07ed (0xbfd0 bytes) 2024-09-14T22:53:11.907 INFO:teuthology.orchestra.run.smithi076.stdout:389 trunc from 0x5125ff to 0x4ae4d6 2024-09-14T22:53:11.916 INFO:teuthology.orchestra.run.smithi076.stdout:390 read 0x395dd thru 0x3df5c (0x4980 bytes) 2024-09-14T22:53:11.916 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.916 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:11.936 INFO:teuthology.orchestra.run.smithi076.stdout:393 write 0xb783d90 thru 0xb789246 (0x54b7 bytes) 2024-09-14T22:53:11.939 INFO:teuthology.orchestra.run.smithi076.stdout:394 read 0x14ee1fa thru 0x14f9223 (0xb02a bytes) 2024-09-14T22:53:11.940 INFO:teuthology.orchestra.run.smithi076.stdout:395 read 0xa74abc2 thru 0xa74c169 (0x15a8 bytes) 2024-09-14T22:53:11.940 INFO:teuthology.orchestra.run.smithi076.stdout:396 read 0x2b6cae thru 0x2c19bf (0xad12 bytes) 2024-09-14T22:53:11.940 INFO:teuthology.orchestra.run.smithi076.stdout:397 write 0x3e152ff thru 0x3e18e11 (0x3b13 bytes) 2024-09-14T22:53:11.942 INFO:teuthology.orchestra.run.smithi076.stdout:398 write 0x3561ebf thru 0x356b610 (0x9752 bytes) 2024-09-14T22:53:11.945 INFO:teuthology.orchestra.run.smithi076.stdout:399 writesame 0xa94ab1a thru 0xa94acfc data_size 0x3(0x1e3 bytes) 2024-09-14T22:53:11.951 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:11.949+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:11.961 INFO:teuthology.orchestra.run.smithi076.stdout:400 trunc from 0xb789247 to 0x17b9668 2024-09-14T22:53:11.994 INFO:teuthology.orchestra.run.smithi076.stdout:401 writesame 0xc743886 thru 0xc7451c7 data_size 0x2(0x1942 bytes) 2024-09-14T22:53:11.994 INFO:teuthology.orchestra.run.smithi076.stdout:402 writesame 0x91a941d thru 0x91b902f data_size 0x1(0xfc13 bytes) 2024-09-14T22:53:11.998 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:11.999 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:12.007 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:12.039 INFO:teuthology.orchestra.run.smithi076.stdout:403 punch from 0x8f86bb8 to 0x8f9580b, (0xec53 bytes) 2024-09-14T22:53:12.040 INFO:teuthology.orchestra.run.smithi076.stdout:406 clone 1 order 20 su 65536 sc 14 2024-09-14T22:53:12.053 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:12.051+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:12.110 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:12.110 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:12.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:12.121 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:12.121 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:12.121 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 28, 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:53:09.759518+0000", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5806080, 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "nodeep-scrub", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "noscrub", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:53:12.122 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:53:12.123 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:12.124 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:12.125 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:12.126 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:12.127 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:12.128 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:53:12.129 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:12.130 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:12.131 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:53:12.132 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:12.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:12.134 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.9900000095367432, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.9900000095367432, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.75, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.75, 2024-09-14T22:53:12.135 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:12.136 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:53:12.137 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "27", 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:53:12.138 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 27, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:12.139 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:12.140 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.5, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.5, 2024-09-14T22:53:12.141 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.5, 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.5, 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:12.142 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:53:12.143 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.144 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:53:12.145 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.146 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:53:12.147 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:12.148 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:12.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.151 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.152 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:53:12.153 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:12.154 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 19, 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:12.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:12.156 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:12.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:53:12.158 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:12.159 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.160 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:53:12.161 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.162 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.163 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:53:12.164 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:12.165 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.166 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:53:12.167 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:12.168 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:12.169 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:53:12.170 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.171 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.172 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.173 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:53:12.175 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:12.176 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 25, 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:12.177 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.178 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.179 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:12.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:53:12.181 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:12.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:12.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.186 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:12.187 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:12.188 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:12.189 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:12.190 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.192 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [], 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [], 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:53:12.193 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [ 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:53:12.194 INFO:teuthology.orchestra.run.smithi076.stdout: "snaps": [ 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: "begin": 2, 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: "length": 1 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:53:12.195 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:53:12.196 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:53:12.196 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:53:12.196 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:53:12.196 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:53:12.196 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.196 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:12.198 INFO:tasks.thrashosds.thrasher:j is {'epoch': 28, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:53:09.759518+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 27, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.0', 'osds': [6, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:53:12.198 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:53:12.199 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:12.195+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:12.298 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:12.296+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:12.300 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:12.300 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:12.309 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:12.368 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:12.398 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:12.397+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:12.452 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:12.452 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:12.460 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:12.490 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:12.491 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:12.501 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:12.500+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:12.509 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:12.509 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:12.525 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:12.526 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:53:12.527 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:12.527 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:12.537 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-14T22:53:10.785792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":150,"num_read_kb":126,"num_write":167,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219692,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755028,"statfs":{"total":773094113280,"available":772869148672,"internally_reserved":0,"allocated":3514368,"data_stored":2036256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039847"},"pg_stats":[{"pgid":"3.1f","version":"27'1","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767167+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:53:09.767167+0000","last_peered":"2024-09-14T22:53:09.767167+0000","last_clean":"2024-09-14T22:53:09.767167+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:53:09.767167+0000","last_undegraded":"2024-09-14T22:53:09.767167+0000","last_fullsized":"2024-09-14T22:53:09.767167+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:05:34.999489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.761968+0000","last_change":"2024-09-14T22:53:08.761968+0000","last_active":"2024-09-14T22:53:08.761968+0000","last_peered":"2024-09-14T22:53:08.761968+0000","last_clean":"2024-09-14T22:53:08.761968+0000","last_became_active":"2024-09-14T22:53:06.755533+0000","last_became_peered":"2024-09-14T22:53:06.755533+0000","last_unstale":"2024-09-14T22:53:08.761968+0000","last_undegraded":"2024-09-14T22:53:08.761968+0000","last_fullsized":"2024-09-14T22:53:08.761968+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:52:42.621981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098261599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.770923+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:53:08.770923+0000","last_peered":"2024-09-14T22:53:08.770923+0000","last_clean":"2024-09-14T22:53:08.770923+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:53:08.770923+0000","last_undegraded":"2024-09-14T22:53:08.770923+0000","last_fullsized":"2024-09-14T22:53:08.770923+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:20:40.608736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"27'1","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.774299+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:53:08.774299+0000","last_peered":"2024-09-14T22:53:08.774299+0000","last_clean":"2024-09-14T22:53:08.774299+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:53:08.774299+0000","last_undegraded":"2024-09-14T22:53:08.774299+0000","last_fullsized":"2024-09-14T22:53:08.774299+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:35:02.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767386+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:53:09.767386+0000","last_peered":"2024-09-14T22:53:09.767386+0000","last_clean":"2024-09-14T22:53:09.767386+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:53:09.767386+0000","last_undegraded":"2024-09-14T22:53:09.767386+0000","last_fullsized":"2024-09-14T22:53:09.767386+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:16.639156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"27'2","reported_seq":24,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767836+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:53:09.767836+0000","last_peered":"2024-09-14T22:53:09.767836+0000","last_clean":"2024-09-14T22:53:09.767836+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:53:09.767836+0000","last_undegraded":"2024-09-14T22:53:09.767836+0000","last_fullsized":"2024-09-14T22:53:09.767836+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:31:02.528942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.766652+0000","last_change":"2024-09-14T22:53:08.766652+0000","last_active":"2024-09-14T22:53:08.766652+0000","last_peered":"2024-09-14T22:53:08.766652+0000","last_clean":"2024-09-14T22:53:08.766652+0000","last_became_active":"2024-09-14T22:53:06.760167+0000","last_became_peered":"2024-09-14T22:53:06.760167+0000","last_unstale":"2024-09-14T22:53:08.766652+0000","last_undegraded":"2024-09-14T22:53:08.766652+0000","last_fullsized":"2024-09-14T22:53:08.766652+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:29:52.528106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.761429+0000","last_change":"2024-09-14T22:53:08.761429+0000","last_active":"2024-09-14T22:53:08.761429+0000","last_peered":"2024-09-14T22:53:08.761429+0000","last_clean":"2024-09-14T22:53:08.761429+0000","last_became_active":"2024-09-14T22:53:06.759353+0000","last_became_peered":"2024-09-14T22:53:06.759353+0000","last_unstale":"2024-09-14T22:53:08.761429+0000","last_undegraded":"2024-09-14T22:53:08.761429+0000","last_fullsized":"2024-09-14T22:53:08.761429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:02:58.988383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062866099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767263+0000","last_change":"2024-09-14T22:53:08.777352+0000","last_active":"2024-09-14T22:53:09.767263+0000","last_peered":"2024-09-14T22:53:09.767263+0000","last_clean":"2024-09-14T22:53:09.767263+0000","last_became_active":"2024-09-14T22:53:06.758502+0000","last_became_peered":"2024-09-14T22:53:06.758502+0000","last_unstale":"2024-09-14T22:53:09.767263+0000","last_undegraded":"2024-09-14T22:53:09.767263+0000","last_fullsized":"2024-09-14T22:53:09.767263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:28.373197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.771175+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:53:08.771175+0000","last_peered":"2024-09-14T22:53:08.771175+0000","last_clean":"2024-09-14T22:53:08.771175+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:53:08.771175+0000","last_undegraded":"2024-09-14T22:53:08.771175+0000","last_fullsized":"2024-09-14T22:53:08.771175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:02:26.572398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829950+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:53:08.829950+0000","last_peered":"2024-09-14T22:53:08.829950+0000","last_clean":"2024-09-14T22:53:08.829950+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:53:08.829950+0000","last_undegraded":"2024-09-14T22:53:08.829950+0000","last_fullsized":"2024-09-14T22:53:08.829950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:56:19.212941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767966+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:53:09.767966+0000","last_peered":"2024-09-14T22:53:09.767966+0000","last_clean":"2024-09-14T22:53:09.767966+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:53:09.767966+0000","last_undegraded":"2024-09-14T22:53:09.767966+0000","last_fullsized":"2024-09-14T22:53:09.767966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:31:32.819368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829465+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:53:08.829465+0000","last_peered":"2024-09-14T22:53:08.829465+0000","last_clean":"2024-09-14T22:53:08.829465+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:53:08.829465+0000","last_undegraded":"2024-09-14T22:53:08.829465+0000","last_fullsized":"2024-09-14T22:53:08.829465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"27'1","reported_seq":22,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767422+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:53:09.767422+0000","last_peered":"2024-09-14T22:53:09.767422+0000","last_clean":"2024-09-14T22:53:09.767422+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:53:09.767422+0000","last_undegraded":"2024-09-14T22:53:09.767422+0000","last_fullsized":"2024-09-14T22:53:09.767422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:55:01.630364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.766425+0000","last_change":"2024-09-14T22:52:59.538561+0000","last_active":"2024-09-14T22:53:08.766425+0000","last_peered":"2024-09-14T22:53:08.766425+0000","last_clean":"2024-09-14T22:53:08.766425+0000","last_became_active":"2024-09-14T22:52:57.530436+0000","last_became_peered":"2024-09-14T22:52:57.530436+0000","last_unstale":"2024-09-14T22:53:08.766425+0000","last_undegraded":"2024-09-14T22:53:08.766425+0000","last_fullsized":"2024-09-14T22:53:08.766425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:29.562282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.762000+0000","last_change":"2024-09-14T22:53:08.762000+0000","last_active":"2024-09-14T22:53:08.762000+0000","last_peered":"2024-09-14T22:53:08.762000+0000","last_clean":"2024-09-14T22:53:08.762000+0000","last_became_active":"2024-09-14T22:53:06.761953+0000","last_became_peered":"2024-09-14T22:53:06.761953+0000","last_unstale":"2024-09-14T22:53:08.762000+0000","last_undegraded":"2024-09-14T22:53:08.762000+0000","last_fullsized":"2024-09-14T22:53:08.762000+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:05:16.337495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829303+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:08.829303+0000","last_peered":"2024-09-14T22:53:08.829303+0000","last_clean":"2024-09-14T22:53:08.829303+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:08.829303+0000","last_undegraded":"2024-09-14T22:53:08.829303+0000","last_fullsized":"2024-09-14T22:53:08.829303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.766672+0000","last_change":"2024-09-14T22:53:08.766672+0000","last_active":"2024-09-14T22:53:08.766672+0000","last_peered":"2024-09-14T22:53:08.766672+0000","last_clean":"2024-09-14T22:53:08.766672+0000","last_became_active":"2024-09-14T22:53:06.755437+0000","last_became_peered":"2024-09-14T22:53:06.755437+0000","last_unstale":"2024-09-14T22:53:08.766672+0000","last_undegraded":"2024-09-14T22:53:08.766672+0000","last_fullsized":"2024-09-14T22:53:08.766672+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:00:20.507639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.766315+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:53:08.766315+0000","last_peered":"2024-09-14T22:53:08.766315+0000","last_clean":"2024-09-14T22:53:08.766315+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:53:08.766315+0000","last_undegraded":"2024-09-14T22:53:08.766315+0000","last_fullsized":"2024-09-14T22:53:08.766315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.771398+0000","last_change":"2024-09-14T22:53:08.771398+0000","last_active":"2024-09-14T22:53:08.771398+0000","last_peered":"2024-09-14T22:53:08.771398+0000","last_clean":"2024-09-14T22:53:08.771398+0000","last_became_active":"2024-09-14T22:53:06.758572+0000","last_became_peered":"2024-09-14T22:53:06.758572+0000","last_unstale":"2024-09-14T22:53:08.771398+0000","last_undegraded":"2024-09-14T22:53:08.771398+0000","last_fullsized":"2024-09-14T22:53:08.771398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.741797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767902+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:53:09.767902+0000","last_peered":"2024-09-14T22:53:09.767902+0000","last_clean":"2024-09-14T22:53:09.767902+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:53:09.767902+0000","last_undegraded":"2024-09-14T22:53:09.767902+0000","last_fullsized":"2024-09-14T22:53:09.767902+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:38:10.208472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.770336+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:53:08.770336+0000","last_peered":"2024-09-14T22:53:08.770336+0000","last_clean":"2024-09-14T22:53:08.770336+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:53:08.770336+0000","last_undegraded":"2024-09-14T22:53:08.770336+0000","last_fullsized":"2024-09-14T22:53:08.770336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829283+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:53:08.829283+0000","last_peered":"2024-09-14T22:53:08.829283+0000","last_clean":"2024-09-14T22:53:08.829283+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:53:08.829283+0000","last_undegraded":"2024-09-14T22:53:08.829283+0000","last_fullsized":"2024-09-14T22:53:08.829283+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:41:32.817978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829038+0000","last_change":"2024-09-14T22:53:00.039871+0000","last_active":"2024-09-14T22:53:08.829038+0000","last_peered":"2024-09-14T22:53:08.829038+0000","last_clean":"2024-09-14T22:53:08.829038+0000","last_became_active":"2024-09-14T22:52:57.694658+0000","last_became_peered":"2024-09-14T22:52:57.694658+0000","last_unstale":"2024-09-14T22:53:08.829038+0000","last_undegraded":"2024-09-14T22:53:08.829038+0000","last_fullsized":"2024-09-14T22:53:08.829038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:09.229943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767204+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:53:09.767204+0000","last_peered":"2024-09-14T22:53:09.767204+0000","last_clean":"2024-09-14T22:53:09.767204+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:53:09.767204+0000","last_undegraded":"2024-09-14T22:53:09.767204+0000","last_fullsized":"2024-09-14T22:53:09.767204+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:13:33.590352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.828966+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:08.828966+0000","last_peered":"2024-09-14T22:53:08.828966+0000","last_clean":"2024-09-14T22:53:08.828966+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:08.828966+0000","last_undegraded":"2024-09-14T22:53:08.828966+0000","last_fullsized":"2024-09-14T22:53:08.828966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.761713+0000","last_change":"2024-09-14T22:53:08.761713+0000","last_active":"2024-09-14T22:53:08.761713+0000","last_peered":"2024-09-14T22:53:08.761713+0000","last_clean":"2024-09-14T22:53:08.761713+0000","last_became_active":"2024-09-14T22:53:06.763198+0000","last_became_peered":"2024-09-14T22:53:06.763198+0000","last_unstale":"2024-09-14T22:53:08.761713+0000","last_undegraded":"2024-09-14T22:53:08.761713+0000","last_fullsized":"2024-09-14T22:53:08.761713+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:08:21.747988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.770232+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:53:08.770232+0000","last_peered":"2024-09-14T22:53:08.770232+0000","last_clean":"2024-09-14T22:53:08.770232+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:53:08.770232+0000","last_undegraded":"2024-09-14T22:53:08.770232+0000","last_fullsized":"2024-09-14T22:53:08.770232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829007+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:08.829007+0000","last_peered":"2024-09-14T22:53:08.829007+0000","last_clean":"2024-09-14T22:53:08.829007+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:08.829007+0000","last_undegraded":"2024-09-14T22:53:08.829007+0000","last_fullsized":"2024-09-14T22:53:08.829007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.789396+0000","last_change":"2024-09-14T22:53:08.761436+0000","last_active":"2024-09-14T22:53:08.789396+0000","last_peered":"2024-09-14T22:53:08.789396+0000","last_clean":"2024-09-14T22:53:08.789396+0000","last_became_active":"2024-09-14T22:53:06.757033+0000","last_became_peered":"2024-09-14T22:53:06.757033+0000","last_unstale":"2024-09-14T22:53:08.789396+0000","last_undegraded":"2024-09-14T22:53:08.789396+0000","last_fullsized":"2024-09-14T22:53:08.789396+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:35:45.234049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767841+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:53:09.767841+0000","last_peered":"2024-09-14T22:53:09.767841+0000","last_clean":"2024-09-14T22:53:09.767841+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:53:09.767841+0000","last_undegraded":"2024-09-14T22:53:09.767841+0000","last_fullsized":"2024-09-14T22:53:09.767841+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:10:52.397491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829619+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:53:08.829619+0000","last_peered":"2024-09-14T22:53:08.829619+0000","last_clean":"2024-09-14T22:53:08.829619+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:53:08.829619+0000","last_undegraded":"2024-09-14T22:53:08.829619+0000","last_fullsized":"2024-09-14T22:53:08.829619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:30:21.551728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.761714+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:53:08.761714+0000","last_peered":"2024-09-14T22:53:08.761714+0000","last_clean":"2024-09-14T22:53:08.761714+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:53:08.761714+0000","last_undegraded":"2024-09-14T22:53:08.761714+0000","last_fullsized":"2024-09-14T22:53:08.761714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:30.448483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.770726+0000","last_change":"2024-09-14T22:53:08.770726+0000","last_active":"2024-09-14T22:53:08.770726+0000","last_peered":"2024-09-14T22:53:08.770726+0000","last_clean":"2024-09-14T22:53:08.770726+0000","last_became_active":"2024-09-14T22:53:06.761991+0000","last_became_peered":"2024-09-14T22:53:06.761991+0000","last_unstale":"2024-09-14T22:53:08.770726+0000","last_undegraded":"2024-09-14T22:53:08.770726+0000","last_fullsized":"2024-09-14T22:53:08.770726+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:47:17.839975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.830212+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:53:08.830212+0000","last_peered":"2024-09-14T22:53:08.830212+0000","last_clean":"2024-09-14T22:53:08.830212+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:53:08.830212+0000","last_undegraded":"2024-09-14T22:53:08.830212+0000","last_fullsized":"2024-09-14T22:53:08.830212+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:48:26.214239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829690+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:53:08.829690+0000","last_peered":"2024-09-14T22:53:08.829690+0000","last_clean":"2024-09-14T22:53:08.829690+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:53:08.829690+0000","last_undegraded":"2024-09-14T22:53:08.829690+0000","last_fullsized":"2024-09-14T22:53:08.829690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:19:32.930404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829329+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:53:08.829329+0000","last_peered":"2024-09-14T22:53:08.829329+0000","last_clean":"2024-09-14T22:53:08.829329+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:53:08.829329+0000","last_undegraded":"2024-09-14T22:53:08.829329+0000","last_fullsized":"2024-09-14T22:53:08.829329+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:20:39.758411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829747+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:53:08.829747+0000","last_peered":"2024-09-14T22:53:08.829747+0000","last_clean":"2024-09-14T22:53:08.829747+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:53:08.829747+0000","last_undegraded":"2024-09-14T22:53:08.829747+0000","last_fullsized":"2024-09-14T22:53:08.829747+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:42:51.456145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829861+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:53:08.829861+0000","last_peered":"2024-09-14T22:53:08.829861+0000","last_clean":"2024-09-14T22:53:08.829861+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:53:08.829861+0000","last_undegraded":"2024-09-14T22:53:08.829861+0000","last_fullsized":"2024-09-14T22:53:08.829861+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:52:25.178267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T22:53:09.767203+0000","last_change":"2024-09-14T22:53:08.776638+0000","last_active":"2024-09-14T22:53:09.767203+0000","last_peered":"2024-09-14T22:53:09.767203+0000","last_clean":"2024-09-14T22:53:09.767203+0000","last_became_active":"2024-09-14T22:53:06.760891+0000","last_became_peered":"2024-09-14T22:53:06.760891+0000","last_unstale":"2024-09-14T22:53:09.767203+0000","last_undegraded":"2024-09-14T22:53:09.767203+0000","last_fullsized":"2024-09-14T22:53:09.767203+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:38:56.537474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T22:53:08.829606+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:53:08.829606+0000","last_peered":"2024-09-14T22:53:08.829606+0000","last_clean":"2024-09-14T22:53:08.829606+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:53:08.829606+0000","last_undegraded":"2024-09-14T22:53:08.829606+0000","last_fullsized":"2024-09-14T22:53:08.829606+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:24.901590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":880640,"data_stored":697287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":106919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":294912,"data_stored":106954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":290816,"data_stored":106938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":106919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":106919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":299008,"data_stored":106987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":888832,"data_stored":697333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":49,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:53:12.539 INFO:tasks.thrashosds.thrasher:Setting 3.e to [2, 3, 6, 0] 2024-09-14T22:53:12.539 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.e', '2', '3', '6', '0'] 2024-09-14T22:53:12.540 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.e 2 3 6 0 2024-09-14T22:53:12.556 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:12.556 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:12.574 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:12.574 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:12.578 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:12.578 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:12.578 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:12.578 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:12.587 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:12.591 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:12.592 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:12.599 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:12.598+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:12.608 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:12.608 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:12.625 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:12.697 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:12.698 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:12.698 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:12.700 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:12.699+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:12.706 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:12.732 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:53:12.802 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:12.800+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.654469+0000", 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 20.167324245, 2024-09-14T22:53:12.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0044819769999999998, 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654469+0000", 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654469+0000", 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654470+0000", 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654478+0000", 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5120000000000008e-06 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654487+0000", 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0890000000000006e-06 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654495+0000", 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4499999999999998e-06 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654543+0000", 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8128999999999999e-05 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.654568+0000", 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5077999999999999e-05 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.658913+0000", 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043456370000000003 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.658951+0000", 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7446e-05 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:18 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'8, mlcod=17'5)", 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.674322+0000", 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 20.147471250999999, 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0040224830000000003, 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674322+0000", 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674322+0000", 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674323+0000", 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5600000000000004e-07 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674339+0000", 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6702999999999999e-05 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674353+0000", 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3286e-05 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674358+0000", 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5779999999999998e-06 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674412+0000", 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3434999999999999e-05 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.674428+0000", 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6073e-05 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.678303+0000", 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0038748609999999998 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.678344+0000", 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1690999999999998e-05 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.832 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.683550+0000", 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 20.138242774999998, 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032167889999999998, 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683550+0000", 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683550+0000", 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683551+0000", 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0200000000000001e-07 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683557+0000", 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4180000000000002e-06 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683565+0000", 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8329999999999994e-06 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683571+0000", 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4070000000000001e-06 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683616+0000", 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5170999999999998e-05 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.683637+0000", 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0761e-05 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.686738+0000", 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003101159 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.686767+0000", 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9337999999999999e-05 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:120 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'82, mlcod=17'78)", 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.992413+0000", 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.829379848999999, 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0029939319999999999, 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.837 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992413+0000", 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992413+0000", 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992414+0000", 2024-09-14T22:53:12.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6799999999999999e-07 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992460+0000", 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6294000000000001e-05 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992472+0000", 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1771e-05 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992477+0000", 2024-09-14T22:53:12.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1719999999999998e-06 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992512+0000", 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4499999999999998e-05 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992525+0000", 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3366e-05 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.995372+0000", 2024-09-14T22:53:12.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002846389 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.995407+0000", 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5571999999999998e-05 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:121 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'83, mlcod=17'78)", 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:52.992798+0000", 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.828994873999999, 2024-09-14T22:53:12.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054025760000000001, 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992798+0000", 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992798+0000", 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992799+0000", 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7199999999999999e-07 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992872+0000", 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2799999999999994e-05 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992882+0000", 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.025e-05 2024-09-14T22:53:12.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992889+0000", 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7129999999999999e-06 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992924+0000", 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5510999999999999e-05 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.992937+0000", 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2490000000000001e-05 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.998170+0000", 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052333060000000001 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:52.998201+0000", 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0933999999999999e-05 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:129 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'87, mlcod=17'85)", 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.383413+0000", 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.438380289000001, 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031314799999999999, 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383413+0000", 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383413+0000", 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383414+0000", 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4700000000000001e-07 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383419+0000", 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1560000000000003e-06 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383429+0000", 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0757000000000001e-05 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383434+0000", 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8960000000000002e-06 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383467+0000", 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2546000000000003e-05 2024-09-14T22:53:12.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383479+0000", 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1987000000000001e-05 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386488+0000", 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030093329999999999 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386544+0000", 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6158e-05 2024-09-14T22:53:12.849 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:130 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'88, mlcod=17'85)", 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.383847+0000", 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.437946584999999, 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030430019999999999, 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383847+0000", 2024-09-14T22:53:12.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383847+0000", 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383847+0000", 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383852+0000", 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7480000000000001e-06 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383862+0000", 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8079999999999999e-06 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383866+0000", 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6319999999999999e-06 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383913+0000", 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7092999999999998e-05 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.383931+0000", 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7428999999999999e-05 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386864+0000", 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0029328290000000001 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386890+0000", 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5876000000000001e-05 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:131 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'89, mlcod=17'85)", 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.384287+0000", 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.43750627, 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054347629999999996, 2024-09-14T22:53:12.854 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384287+0000", 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384287+0000", 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384287+0000", 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7999999999999995e-07 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384313+0000", 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5525e-05 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384323+0000", 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0379e-05 2024-09-14T22:53:12.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384328+0000", 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7060000000000003e-06 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384364+0000", 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6229999999999997e-05 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384375+0000", 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1080000000000001e-05 2024-09-14T22:53:12.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389686+0000", 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053104720000000001 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389722+0000", 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5790999999999997e-05 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.858 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:132 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'90, mlcod=17'85)", 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.384792+0000", 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.437000756, 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0049634750000000002, 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384792+0000", 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384792+0000", 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384793+0000", 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.82e-07 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384798+0000", 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5249999999999999e-06 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384808+0000", 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0200999999999999e-05 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384812+0000", 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384848+0000", 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5438999999999997e-05 2024-09-14T22:53:12.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.384860+0000", 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.201e-05 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389725+0000", 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048650940000000004 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389756+0000", 2024-09-14T22:53:12.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0942000000000003e-05 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:133 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'91, mlcod=17'85)", 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.385363+0000", 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.436429733000001, 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00442678, 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.863 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385363+0000", 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385363+0000", 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385364+0000", 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5000000000000002e-07 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385369+0000", 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7890000000000002e-06 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385379+0000", 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0540000000000001e-05 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385384+0000", 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6569999999999996e-06 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385423+0000", 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8693999999999999e-05 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385434+0000", 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1309e-05 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389757+0000", 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043230609999999996 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389790+0000", 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3080000000000002e-05 2024-09-14T22:53:12.867 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:134 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'92, mlcod=17'85)", 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.385819+0000", 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.435974342000002, 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0040019410000000002, 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385819+0000", 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385819+0000", 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385819+0000", 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1899999999999998e-07 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.869 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385883+0000", 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3774000000000003e-05 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385890+0000", 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9349999999999997e-06 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385895+0000", 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2889999999999997e-06 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385929+0000", 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3983999999999999e-05 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.385942+0000", 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3274e-05 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389791+0000", 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0038488060000000002 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389821+0000", 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9560000000000002e-05 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:135 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'93, mlcod=17'85)", 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.386482+0000", 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.435310961999999, 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0033692790000000002, 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.872 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386482+0000", 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386482+0000", 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386483+0000", 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386488+0000", 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.763e-06 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386500+0000", 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1883e-05 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386509+0000", 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6579999999999997e-06 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386568+0000", 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8477999999999998e-05 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386581+0000", 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3808999999999999e-05 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389823+0000", 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003241318 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389851+0000", 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8685999999999999e-05 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:136 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'94, mlcod=17'85)", 2024-09-14T22:53:12.876 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.386959+0000", 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.434834597999998, 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0029261230000000001, 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386959+0000", 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386959+0000", 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386959+0000", 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.99e-07 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386963+0000", 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3390000000000003e-06 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386971+0000", 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6429999999999995e-06 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.386977+0000", 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.891e-06 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.387008+0000", 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1343999999999997e-05 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.387019+0000", 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0633e-05 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389853+0000", 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0028337729999999999 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.389885+0000", 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1900999999999998e-05 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.880 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'98)", 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.393598+0000", 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.428195402, 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0024712050000000002, 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393598+0000", 2024-09-14T22:53:12.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393598+0000", 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393598+0000", 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393664+0000", 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5845000000000004e-05 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393671+0000", 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1930000000000004e-06 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393675+0000", 2024-09-14T22:53:12.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7249999999999999e-06 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393707+0000", 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2224999999999997e-05 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.393720+0000", 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3132999999999999e-05 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.396038+0000", 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0023174100000000002 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.396069+0000", 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1327e-05 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:155 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=17'105)", 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.844740+0000", 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.977053507000001, 2024-09-14T22:53:12.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002873914, 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844740+0000", 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844740+0000", 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844740+0000", 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4999999999999999e-07 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844819+0000", 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9184999999999995e-05 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844827+0000", 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6129999999999998e-06 2024-09-14T22:53:12.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844832+0000", 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4140000000000002e-06 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844862+0000", 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9774e-05 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.844876+0000", 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3794e-05 2024-09-14T22:53:12.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847572+0000", 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0026964110000000001 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847614+0000", 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1473000000000001e-05 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.889 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:156 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'110, mlcod=17'105)", 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.845407+0000", 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.976386548000001, 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0046289319999999997, 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845407+0000", 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845407+0000", 2024-09-14T22:53:12.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845407+0000", 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845412+0000", 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4830000000000001e-06 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845419+0000", 2024-09-14T22:53:12.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.956e-06 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845425+0000", 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845456+0000", 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0837000000000003e-05 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.845467+0000", 2024-09-14T22:53:12.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1419e-05 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.849994+0000", 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0045264190000000003 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850036+0000", 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1757e-05 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.893 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:157 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'111, mlcod=17'107)", 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.846220+0000", 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.975573580999999, 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003845677, 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846220+0000", 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846220+0000", 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846220+0000", 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6100000000000001e-07 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846224+0000", 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.827e-06 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846231+0000", 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.063e-06 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846234+0000", 2024-09-14T22:53:12.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4659999999999999e-06 2024-09-14T22:53:12.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846263+0000", 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8802999999999998e-05 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846276+0000", 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2714999999999999e-05 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850038+0000", 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0037620549999999998 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850065+0000", 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7186999999999999e-05 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.899 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:158 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=17'107)", 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.846753+0000", 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.975040432, 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0033415490000000001, 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846753+0000", 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846753+0000", 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846753+0000", 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846820+0000", 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7288999999999999e-05 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846827+0000", 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3640000000000004e-06 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846830+0000", 2024-09-14T22:53:12.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1e-06 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846856+0000", 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999999e-05 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.846867+0000", 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1429e-05 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850066+0000", 2024-09-14T22:53:12.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031991089999999999 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850094+0000", 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7898e-05 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4195.0:159 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'113, mlcod=17'107)", 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:53.847344+0000", 2024-09-14T22:53:12.904 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.974449257, 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0029449179999999999, 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847344+0000", 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847344+0000", 2024-09-14T22:53:12.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847344+0000", 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9999999999999998e-07 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847348+0000", 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7720000000000001e-06 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847356+0000", 2024-09-14T22:53:12.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6459999999999993e-06 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847359+0000", 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1360000000000001e-06 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847395+0000", 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6647999999999999e-05 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.847408+0000", 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2130000000000001e-05 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850265+0000", 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0028577810000000002 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:53.850289+0000", 2024-09-14T22:53:12.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3405e-05 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.774468+0000", 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.0473252909999999, 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032723750000000001, 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:12.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 11 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774468+0000", 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774468+0000", 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:12.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774470+0000", 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7460000000000001e-06 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774478+0000", 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4570000000000005e-06 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774493+0000", 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5347999999999999e-05 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774502+0000", 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2910000000000007e-06 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774568+0000", 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6527999999999997e-05 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.774573+0000", 2024-09-14T22:53:12.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6500000000000004e-06 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.776325+0000", 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017522810000000001 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.776464+0000", 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000139326 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 5", 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.776684+0000", 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021937399999999999 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.777590+0000", 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00090645999999999999 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.777679+0000", 2024-09-14T22:53:12.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8632000000000001e-05 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.777710+0000", 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0639000000000003e-05 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.777740+0000", 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0642999999999998e-05 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.915 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.916 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.916 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:12.916 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:12.916 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:12.917 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:12.915+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:12.917 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:12.922 INFO:teuthology.orchestra.run.smithi076.stdout:408 punch from 0x37afde4 to 0x37bac01, (0xae1d bytes) 2024-09-14T22:53:12.925 INFO:teuthology.orchestra.run.smithi076.stdout:409 read 0x6a2db36 thru 0x6a394f3 (0xb9be bytes) 2024-09-14T22:53:12.925 INFO:teuthology.orchestra.run.smithi076.stdout:410 read 0x43ce803 thru 0x43d775b (0x8f59 bytes) 2024-09-14T22:53:12.926 INFO:teuthology.orchestra.run.smithi076.stdout:411 read 0xaeab6c9 thru 0xaeb3553 (0x7e8b bytes) 2024-09-14T22:53:12.926 INFO:teuthology.orchestra.run.smithi076.stdout:412 writesame 0x74aba64 thru 0x74ac38d data_size 0x22(0x92a bytes) 2024-09-14T22:53:12.929 INFO:teuthology.orchestra.run.smithi076.stdout:414 punch from 0x2a05f10 to 0x2a0f321, (0x9411 bytes) 2024-09-14T22:53:12.929 INFO:teuthology.orchestra.run.smithi076.stdout:415 writesame 0xac5c7b2 thru 0xac660b0 data_size 0x1(0x98ff bytes) 2024-09-14T22:53:12.953 INFO:teuthology.orchestra.run.smithi076.stdout:417 write 0x7d3288b thru 0x7d3eaef (0xc265 bytes) 2024-09-14T22:53:12.957 INFO:teuthology.orchestra.run.smithi076.stdout:418 write 0x840df9c thru 0x841a4b2 (0xc517 bytes) 2024-09-14T22:53:12.961 INFO:teuthology.orchestra.run.smithi076.stdout:419 write 0x8b0fa1 thru 0x8badef (0x9e4f bytes) 2024-09-14T22:53:12.973 INFO:teuthology.orchestra.run.smithi076.stdout:421 trunc from 0xc7451c8 to 0x6287f53 2024-09-14T22:53:13.011 INFO:teuthology.orchestra.run.smithi076.stdout:422 write 0xb843eb7 thru 0xb8487f0 (0x493a bytes) 2024-09-14T22:53:13.017 INFO:teuthology.orchestra.run.smithi076.stdout:423 write 0x84e47bb thru 0x84f21d0 (0xda16 bytes) 2024-09-14T22:53:13.018 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:13.016+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:13.022 INFO:teuthology.orchestra.run.smithi076.stdout:424 write 0x939f79b thru 0x93a06a3 (0xf09 bytes) 2024-09-14T22:53:13.025 INFO:teuthology.orchestra.run.smithi076.stdout:427 read 0x2ea427f thru 0x2eb33d7 (0xf159 bytes) 2024-09-14T22:53:13.026 INFO:teuthology.orchestra.run.smithi076.stdout:429 read 0x9f14f8f thru 0x9f17ec1 (0x2f33 bytes) 2024-09-14T22:53:13.026 INFO:teuthology.orchestra.run.smithi076.stdout:430 read 0x6b7a779 thru 0x6b85350 (0xabd8 bytes) 2024-09-14T22:53:13.038 INFO:teuthology.orchestra.run.smithi076.stdout:431 trunc from 0xb8487f1 to 0x268dc1b 2024-09-14T22:53:13.053 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:13.053 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:13.053 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:13.053 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:13.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:13.072 INFO:teuthology.orchestra.run.smithi076.stdout:432 punch from 0x1e66732 to 0x1e67d8f, (0x165d bytes) 2024-09-14T22:53:13.072 INFO:teuthology.orchestra.run.smithi076.stdout:433 write 0x1511e18 thru 0x1515fa3 (0x418c bytes) 2024-09-14T22:53:13.074 INFO:teuthology.orchestra.run.smithi076.stdout:434 read 0x18d76da thru 0x18e09fc (0x9323 bytes) 2024-09-14T22:53:13.083 INFO:teuthology.orchestra.run.smithi076.stdout:436 write 0x548b01a thru 0x5494d48 (0x9d2f bytes) 2024-09-14T22:53:13.091 INFO:teuthology.orchestra.run.smithi076.stdout:438 read 0x52753db thru 0x527d485 (0x80ab bytes) 2024-09-14T22:53:13.091 INFO:teuthology.orchestra.run.smithi076.stdout:440 writesame 0x29c7596 thru 0x29d27bb data_size 0x21(0xb226 bytes) 2024-09-14T22:53:13.094 INFO:teuthology.orchestra.run.smithi076.stdout:441 write 0x379b4b thru 0x37eb8a (0x5040 bytes) 2024-09-14T22:53:13.096 INFO:teuthology.orchestra.run.smithi076.stdout:442 read 0x5098425 thru 0x509e67c (0x6258 bytes) 2024-09-14T22:53:13.096 INFO:teuthology.orchestra.run.smithi076.stdout:443 read 0xfb38f7 thru 0xfb5fe7 (0x26f1 bytes) 2024-09-14T22:53:13.110 INFO:teuthology.orchestra.run.smithi076.stdout:444 write 0xb99563c thru 0xb998998 (0x335d bytes) 2024-09-14T22:53:13.113 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:13.113 INFO:teuthology.orchestra.run.smithi076.stdout:447 write 0x35aa4b3 thru 0x35b1482 (0x6fd0 bytes) 2024-09-14T22:53:13.118 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:13.117+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:13.118 INFO:teuthology.orchestra.run.smithi076.stdout:448 write 0x824f8ec thru 0x82587a9 (0x8ebe bytes) 2024-09-14T22:53:13.121 INFO:teuthology.orchestra.run.smithi076.stdout:449 read 0x3ec7139 thru 0x3ed62c9 (0xf191 bytes) 2024-09-14T22:53:13.122 INFO:teuthology.orchestra.run.smithi076.stdout:450 read 0xe17070 thru 0xe1db3b (0x6acc bytes) 2024-09-14T22:53:13.122 INFO:teuthology.orchestra.run.smithi076.stdout:451 read 0x7333e55 thru 0x7334b08 (0xcb4 bytes) 2024-09-14T22:53:13.122 INFO:teuthology.orchestra.run.smithi076.stdout:453 write 0x16e2e1c thru 0x16e86fd (0x58e2 bytes) 2024-09-14T22:53:13.125 INFO:teuthology.orchestra.run.smithi076.stdout:454 write 0x96a699f thru 0x96ae4a7 (0x7b09 bytes) 2024-09-14T22:53:13.127 INFO:teuthology.orchestra.run.smithi076.stdout:455 write 0x5132603 thru 0x51340d9 (0x1ad7 bytes) 2024-09-14T22:53:13.130 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:13.130 INFO:teuthology.orchestra.run.smithi076.stdout:458 punch from 0x82675d8 to 0x8269b48, (0x2570 bytes) 2024-09-14T22:53:13.130 INFO:teuthology.orchestra.run.smithi076.stdout:459 read 0xa910508 thru 0xa91d645 (0xd13e bytes) 2024-09-14T22:53:13.130 INFO:teuthology.orchestra.run.smithi076.stdout:460 read 0xa569239 thru 0xa56da08 (0x47d0 bytes) 2024-09-14T22:53:13.131 INFO:teuthology.orchestra.run.smithi076.stdout:462 trunc from 0xb998999 to 0xbcc4a01 2024-09-14T22:53:13.137 INFO:teuthology.orchestra.run.smithi076.stdout:463 read 0x11e638c thru 0x11ee17d (0x7df2 bytes) 2024-09-14T22:53:13.137 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:13.137 INFO:teuthology.orchestra.run.smithi076.stdout:465 punch from 0xb41e6fe to 0xb421b22, (0x3424 bytes) 2024-09-14T22:53:13.137 INFO:teuthology.orchestra.run.smithi076.stdout:466 punch from 0x26461ea to 0x26547c2, (0xe5d8 bytes) 2024-09-14T22:53:13.137 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:13.137 INFO:teuthology.orchestra.run.smithi076.stdout:468 read 0xb326b4f thru 0xb32e0dd (0x758f bytes) 2024-09-14T22:53:13.137 INFO:teuthology.orchestra.run.smithi076.stdout:469 read 0xb90e697 thru 0xb914215 (0x5b7f bytes) 2024-09-14T22:53:13.137 INFO:teuthology.orchestra.run.smithi076.stdout:471 write 0x323048e thru 0x323a1ec (0x9d5f bytes) 2024-09-14T22:53:13.138 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:13.147 INFO:teuthology.orchestra.run.smithi076.stdout:474 write 0xd0a2f84 thru 0xd0ae1fb (0xb278 bytes) 2024-09-14T22:53:13.151 INFO:teuthology.orchestra.run.smithi076.stdout:475 write 0xcf31427 thru 0xcf3a89e (0x9478 bytes) 2024-09-14T22:53:13.215 INFO:teuthology.orchestra.run.smithi076.stdout:479 punch from 0x31bf908 to 0x31cad59, (0xb451 bytes) 2024-09-14T22:53:13.215 INFO:teuthology.orchestra.run.smithi076.stdout:480 trunc from 0xd0ae1fc to 0x45c6e79 2024-09-14T22:53:13.215 INFO:teuthology.orchestra.run.smithi076.stdout:481 read 0x198219b thru 0x1992158 (0xffbe bytes) 2024-09-14T22:53:13.215 INFO:teuthology.orchestra.run.smithi076.stdout:484 clone 2 order 18 su 65536 sc 6 2024-09-14T22:53:13.216 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:13.216 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:13.216 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:13.216 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:13.216 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:13.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:13.257 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:13.256+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:13.320 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:13.319+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:13.360 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4232.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:58.530337+0000", 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.828019283, 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00088297699999999996, 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.361 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530337+0000", 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530337+0000", 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530337+0000", 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.99e-07 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530344+0000", 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1650000000000003e-06 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530366+0000", 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2651999999999999e-05 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530379+0000", 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2789000000000001e-05 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530446+0000", 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6827000000000002e-05 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.530473+0000", 2024-09-14T22:53:13.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6874000000000001e-05 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.531172+0000", 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069956900000000004 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:58.531220+0000", 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7902000000000003e-05 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.365 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4232.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:59.536346+0000", 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 13.822010944000001, 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010693110000000001, 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536346+0000", 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536346+0000", 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536346+0000", 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8200000000000001e-07 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536361+0000", 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5047e-05 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536390+0000", 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8568000000000002e-05 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536403+0000", 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3142999999999999e-05 2024-09-14T22:53:13.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536469+0000", 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5920999999999994e-05 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.536497+0000", 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8124e-05 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.537378+0000", 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00088145900000000002 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:59.537415+0000", 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6667e-05 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:9 3.1a e27/24 3:5cea7035:::rbd_info:head v 27'2, mlcod=0'0)", 2024-09-14T22:53:13.370 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.764994+0000", 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.593361979, 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000716671, 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.764994+0000", 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.764994+0000", 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.764995+0000", 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.92e-07 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765000+0000", 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3990000000000003e-06 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765008+0000", 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0960000000000001e-06 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765016+0000", 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0039999999999999e-06 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765059+0000", 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2378e-05 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765075+0000", 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6375000000000002e-05 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765679+0000", 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00060367199999999998 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.765711+0000", 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2555000000000003e-05 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:13 3.6 e27/24 3:62ea7de9:::rbd_object_map.11352933567d:head v 27'1, mlcod=0'0)", 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.780224+0000", 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.5781325229999998, 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069506199999999998, 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.375 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780224+0000", 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780224+0000", 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780224+0000", 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7000000000000001e-07 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780229+0000", 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780245+0000", 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5610000000000001e-05 2024-09-14T22:53:13.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780251+0000", 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6680000000000004e-06 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780305+0000", 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-05 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780324+0000", 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9845000000000001e-05 2024-09-14T22:53:13.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780891+0000", 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654399999999999 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.780919+0000", 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8031e-05 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:25 3.8 e27/24 3:15b9a59a:::rbd_header.11352933567d:head v 27'2, mlcod=0'0)", 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.788529+0000", 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.5698278610000003, 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00072540699999999996, 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788529+0000", 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788529+0000", 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788530+0000", 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.517e-06 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788542+0000", 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1609e-05 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788561+0000", 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9051000000000001e-05 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788571+0000", 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9389999999999992e-06 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788640+0000", 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9671000000000004e-05 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788663+0000", 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2589e-05 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.789217+0000", 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00055367399999999998 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.789254+0000", 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7357000000000002e-05 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:966 3.8 e29/24 3:15b9a59a:::rbd_header.11352933567d:head v 29'82, mlcod=28'80)", 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.736388+0000", 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.62196815400000005, 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0025016700000000001, 2024-09-14T22:53:13.384 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736388+0000", 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736388+0000", 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736389+0000", 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1899999999999999e-07 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736392+0000", 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6849999999999999e-06 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736400+0000", 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.391e-06 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736409+0000", 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2189999999999993e-06 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736454+0000", 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5011e-05 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736476+0000", 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2158e-05 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.738836+0000", 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0023599789999999999 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.738890+0000", 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4008000000000002e-05 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:969 3.6 e29/24 3:62ea7de9:::rbd_object_map.11352933567d:head v 29'195, mlcod=28'193)", 2024-09-14T22:53:13.388 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.748025+0000", 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.61033161499999999, 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00080113700000000005, 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748025+0000", 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748025+0000", 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748025+0000", 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748032+0000", 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2999999999999998e-06 2024-09-14T22:53:13.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748043+0000", 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1532000000000001e-05 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748057+0000", 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4252e-05 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748120+0000", 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2593000000000002e-05 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748142+0000", 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2310000000000002e-05 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748795+0000", 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00065314099999999999 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.748826+0000", 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0568000000000003e-05 2024-09-14T22:53:13.392 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:980 3.8 e29/24 3:15b9a59a:::rbd_header.11352933567d:head v 29'83, mlcod=28'81)", 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.752813+0000", 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.60554297499999998, 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00073741900000000001, 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752813+0000", 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752813+0000", 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752814+0000", 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8999999999999995e-07 2024-09-14T22:53:13.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752821+0000", 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7510000000000002e-06 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752832+0000", 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0587e-05 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752837+0000", 2024-09-14T22:53:13.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2519999999999999e-06 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752895+0000", 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7911999999999997e-05 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752920+0000", 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4697000000000001e-05 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.753520+0000", 2024-09-14T22:53:13.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00060057999999999997 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.753551+0000", 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0750000000000002e-05 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1071 3.19 3:9b34d9d3:::rbd_data.1135de237b71.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 375396~2346] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.927025+0000", 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.43133158799999999, 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00179941, 2024-09-14T22:53:13.397 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1071 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927025+0000", 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927025+0000", 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927027+0000", 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.649e-06 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927037+0000", 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0118e-05 2024-09-14T22:53:13.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927052+0000", 2024-09-14T22:53:13.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5398999999999999e-05 2024-09-14T22:53:13.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927059+0000", 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5520000000000001e-06 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927122+0000", 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3897000000000002e-05 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927256+0000", 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013323499999999999 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.927477+0000", 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022166300000000001 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.928293+0000", 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00081530199999999995 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.928717+0000", 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000424141 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.928781+0000", 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3820999999999998e-05 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.928824+0000", 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3633000000000001e-05 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1091 3.19 3:9b34d9d3:::rbd_data.1135de237b71.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:13.403 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.976557+0000", 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.38179902900000001, 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0013352870000000001, 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1091 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976557+0000", 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976557+0000", 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976559+0000", 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.813e-06 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976566+0000", 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2870000000000001e-06 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976582+0000", 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6422e-05 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976592+0000", 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5969999999999998e-06 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976681+0000", 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9685999999999995e-05 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976774+0000", 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2639999999999994e-05 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.976891+0000", 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000117201 2024-09-14T22:53:13.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.977502+0000", 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00061109000000000001 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.977799+0000", 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029718000000000002 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.977850+0000", 2024-09-14T22:53:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0407e-05 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.977893+0000", 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2963999999999997e-05 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "pg_update_log_missing(3.6 epoch 29/24 rep_tid 251 entries 29'197 (0'0) error 3:643a2945:::rbd_data.1135de237b71.000000000000006f:head by client.4405.0:1109 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 29'195)", 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.980483+0000", 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.37787348999999998, 2024-09-14T22:53:13.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00092659299999999997, 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.980483+0000", 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.980483+0000", 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.980484+0000", 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1699999999999997e-07 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.980492+0000", 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1410000000000005e-06 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.980510+0000", 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8110000000000001e-05 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.980520+0000", 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.499e-06 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.980591+0000", 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1611000000000003e-05 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.981410+0000", 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00081841499999999996 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1112 3.1a e29/24 3:5a42f79b:::rbd_data.1135de237b71.0000000000000063:head v 29'3, mlcod=26'1)", 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.983923+0000", 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.37443378999999999, 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00060893199999999996, 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.413 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.983923+0000", 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.983923+0000", 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.983924+0000", 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.33e-07 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.983931+0000", 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0079999999999996e-06 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.983945+0000", 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4005000000000001e-05 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.983951+0000", 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.983996+0000", 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5021000000000001e-05 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.984013+0000", 2024-09-14T22:53:13.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6121999999999999e-05 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.984497+0000", 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048487200000000002 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.984532+0000", 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4190000000000003e-05 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1116 3.8 e29/24 3:107554a9:::rbd_data.1135de237b71.0000000000000067:head v 29'86, mlcod=29'84)", 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.987344+0000", 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.371012063, 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00063981599999999997, 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987344+0000", 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987344+0000", 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987345+0000", 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3600000000000003e-07 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987352+0000", 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4509999999999997e-06 2024-09-14T22:53:13.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987366+0000", 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4042999999999999e-05 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987373+0000", 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8950000000000001e-06 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987433+0000", 2024-09-14T22:53:13.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0636e-05 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987456+0000", 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3292999999999999e-05 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987944+0000", 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048787 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987984+0000", 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9891999999999997e-05 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1150 3.1a e29/24 3:5a42f79b:::rbd_data.1135de237b71.0000000000000063:head v 29'4, mlcod=27'2)", 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.040211+0000", 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.31814521699999998, 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00072550900000000005, 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.422 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040211+0000", 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040211+0000", 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040212+0000", 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.161e-06 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040223+0000", 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0380999999999999e-05 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040239+0000", 2024-09-14T22:53:13.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5933000000000001e-05 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040247+0000", 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6240000000000008e-06 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040306+0000", 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9132000000000002e-05 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040327+0000", 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0574999999999999e-05 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040894+0000", 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056725999999999999 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040937+0000", 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2443e-05 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.426 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1154 3.8 e29/24 3:107554a9:::rbd_data.1135de237b71.0000000000000067:head v 29'88, mlcod=29'86)", 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.040212+0000", 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.31814482599999999, 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00095124300000000005, 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040212+0000", 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040212+0000", 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040213+0000", 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9199999999999999e-07 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040223+0000", 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0319999999999999e-05 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040242+0000", 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9338999999999999e-05 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040250+0000", 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7729999999999999e-06 2024-09-14T22:53:13.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040315+0000", 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4826999999999994e-05 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040336+0000", 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0698000000000001e-05 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041124+0000", 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00078846000000000005 2024-09-14T22:53:13.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041163+0000", 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8834000000000001e-05 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1179 3.4 3:226f6f0f:::rbd_data.1135de237b71.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.052829+0000", 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.30552786999999998, 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001533815, 2024-09-14T22:53:13.431 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1179 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.052829+0000", 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.052829+0000", 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.052831+0000", 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9489999999999999e-06 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.052837+0000", 2024-09-14T22:53:13.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6889999999999998e-06 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.052854+0000", 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6412000000000001e-05 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.052870+0000", 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6837000000000002e-05 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.052932+0000", 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1948999999999999e-05 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053082+0000", 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014925499999999999 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 2", 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053277+0000", 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019573999999999999 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053969+0000", 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000691579 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.054284+0000", 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031534500000000002 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.054319+0000", 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4870000000000003e-05 2024-09-14T22:53:13.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.054362+0000", 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3189999999999998e-05 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1186 3.4 3:2773f73b:::rbd_data.1135de237b71.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.060906+0000", 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.297450251, 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001536789, 2024-09-14T22:53:13.437 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1186 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060906+0000", 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060906+0000", 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060908+0000", 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.097e-06 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060918+0000", 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2099999999999999e-06 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060936+0000", 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8805999999999999e-05 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060946+0000", 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6889999999999999e-06 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061020+0000", 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4362000000000004e-05 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061152+0000", 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000131626 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 2", 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061328+0000", 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000176322 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061876+0000", 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000547939 2024-09-14T22:53:13.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062352+0000", 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00047527399999999998 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062405+0000", 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3217000000000002e-05 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062443+0000", 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8247000000000002e-05 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1202 3.4 3:2292a349:::rbd_data.1135de237b71.000000000000000f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 531992~16780] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.073044+0000", 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.28531205799999998, 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001319314, 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:13.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1202 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073044+0000", 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073044+0000", 2024-09-14T22:53:13.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073046+0000", 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1799999999999999e-06 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073069+0000", 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3808e-05 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073082+0000", 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.223e-05 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073087+0000", 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8289999999999996e-06 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073131+0000", 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3661000000000001e-05 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073222+0000", 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1378999999999994e-05 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 2", 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073337+0000", 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000114339 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073961+0000", 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00062407200000000004 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.074289+0000", 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00032808000000000002 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.074324+0000", 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4555999999999999e-05 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.074364+0000", 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0179999999999998e-05 2024-09-14T22:53:13.448 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1242 3.1a e29/24 3:5dc85408:::rbd_data.1135de237b71.000000000000008e:head v 29'5, mlcod=29'3)", 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.126270+0000", 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.23208688299999999, 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00080743199999999996, 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.126270+0000", 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.126270+0000", 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.126271+0000", 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0440000000000001e-06 2024-09-14T22:53:13.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.126325+0000", 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3977000000000002e-05 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.126338+0000", 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2951e-05 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.126345+0000", 2024-09-14T22:53:13.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4109999999999999e-06 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.126391+0000", 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5868000000000002e-05 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.126407+0000", 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6444999999999999e-05 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.127040+0000", 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00063310999999999999 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.127077+0000", 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6625999999999998e-05 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.453 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1274 3.1a e29/24 3:5dc85408:::rbd_data.1135de237b71.000000000000008e:head v 29'6, mlcod=29'4)", 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.167976+0000", 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.19038034300000001, 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001405192, 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167976+0000", 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167976+0000", 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167977+0000", 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.319e-06 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167989+0000", 2024-09-14T22:53:13.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1916000000000001e-05 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.168008+0000", 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8658e-05 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.168018+0000", 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6870000000000006e-06 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.168081+0000", 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3075000000000002e-05 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.168098+0000", 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6986999999999998e-05 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169335+0000", 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001236914 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169381+0000", 2024-09-14T22:53:13.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6635999999999999e-05 2024-09-14T22:53:13.458 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.458 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.458 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.458 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.458 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.458 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:13.459 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:13.460 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:13.458+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:13.561 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:13.559+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:13.563 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:13.563 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:13.571 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:13.662 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:13.660+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:13.674 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:13.674 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:13.674 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:13.674 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:13.674 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:13.682 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:13.763 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:13.762+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:13.770 INFO:teuthology.orchestra.run.smithi076.stderr:set 3.e pg_upmap_items mapping to [2->3,6->0] 2024-09-14T22:53:13.789 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:13.789 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:13.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:13.789 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:13.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.789 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4232.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T22:53:13.789 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:13.789 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.257551178, 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010310849999999999, 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8500000000000002e-07 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530823+0000", 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0277e-05 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530835+0000", 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1459e-05 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530845+0000", 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0239e-05 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530907+0000", 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1981000000000002e-05 2024-09-14T22:53:13.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530934+0000", 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6701999999999999e-05 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.531810+0000", 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00087597399999999996 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.531844+0000", 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4068e-05 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1179 3.4 e29/24 3:226f6f0f:::rbd_data.1135de237b71.000000000000001d:head v 29'5, mlcod=28'3)", 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.053425+0000", 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.73493875600000003, 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00075315400000000002, 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.794 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053425+0000", 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053425+0000", 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053425+0000", 2024-09-14T22:53:13.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1600000000000002e-07 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053434+0000", 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.405e-06 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053459+0000", 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5199e-05 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053471+0000", 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2252000000000001e-05 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053527+0000", 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5797000000000001e-05 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053548+0000", 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0908999999999999e-05 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.054134+0000", 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058633100000000005 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.054178+0000", 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3945e-05 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1186 3.4 e29/24 3:2773f73b:::rbd_data.1135de237b71.0000000000000024:head v 29'6, mlcod=28'4)", 2024-09-14T22:53:13.798 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.061416+0000", 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.726947381, 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00078209299999999998, 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061416+0000", 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061416+0000", 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061418+0000", 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1990000000000001e-06 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061428+0000", 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0706e-05 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061447+0000", 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8989999999999999e-05 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061460+0000", 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3176999999999999e-05 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061598+0000", 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013775699999999999 2024-09-14T22:53:13.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061620+0000", 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1328999999999999e-05 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062159+0000", 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00053960300000000004 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062198+0000", 2024-09-14T22:53:13.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9332000000000002e-05 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1202 3.4 e29/24 3:2292a349:::rbd_data.1135de237b71.000000000000000f:head v 29'7, mlcod=29'5)", 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.073403+0000", 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.71496027600000001, 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00078782099999999996, 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:13.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073403+0000", 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073403+0000", 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073405+0000", 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:53:13.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073441+0000", 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6032999999999999e-05 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073457+0000", 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6387000000000001e-05 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073465+0000", 2024-09-14T22:53:13.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0309999999999998e-06 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073522+0000", 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6371e-05 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073545+0000", 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3428999999999999e-05 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:13.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.074162+0000", 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00061683599999999995 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.074191+0000", 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9522999999999998e-05 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:13.807 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:13.808 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:13.835 INFO:teuthology.orchestra.run.smithi076.stdout:leaving image image_client.0-clone1 intact 2024-09-14T22:53:13.878 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:13.877+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:13.911 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T22:53:13.950 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:13.950 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:13.950 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:13.950 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:13.963 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:14.003 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:14.002+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:14.066 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:14.065+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:14.112 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:14.112 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:14.121 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:14.179 INFO:teuthology.orchestra.run.smithi076.stdout:485 trunc from 0x45c6e79 to 0x7c71772 2024-09-14T22:53:14.179 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:14.178+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:14.188 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:14.188 INFO:teuthology.orchestra.run.smithi076.stdout:489 write 0x32b9e45 thru 0x32c6631 (0xc7ed bytes) 2024-09-14T22:53:14.192 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:14.192 INFO:teuthology.orchestra.run.smithi076.stdout:492 writesame 0x54a738b thru 0x54b1683 data_size 0x1(0xa2f9 bytes) 2024-09-14T22:53:14.221 INFO:teuthology.orchestra.run.smithi076.stdout:493 read 0x120822a thru 0x120b6e8 (0x34bf bytes) 2024-09-14T22:53:14.221 INFO:teuthology.orchestra.run.smithi076.stdout:494 read 0x3343603 thru 0x334c139 (0x8b37 bytes) 2024-09-14T22:53:14.221 INFO:teuthology.orchestra.run.smithi076.stdout:496 read 0x2bbf381 thru 0x2bcd0f7 (0xdd77 bytes) 2024-09-14T22:53:14.221 INFO:teuthology.orchestra.run.smithi076.stdout:498 punch from 0x3958bfd to 0x395e752, (0x5b55 bytes) 2024-09-14T22:53:14.221 INFO:teuthology.orchestra.run.smithi076.stdout:499 read 0x40edeec thru 0x40f5c0c (0x7d21 bytes) 2024-09-14T22:53:14.222 INFO:teuthology.orchestra.run.smithi076.stdout:500 read 0x493d1a8 thru 0x4943f0b (0x6d64 bytes) 2024-09-14T22:53:14.222 INFO:teuthology.orchestra.run.smithi076.stdout:502 read 0x7a68aa thru 0x7b42ea (0xda41 bytes) 2024-09-14T22:53:14.222 INFO:teuthology.orchestra.run.smithi076.stdout:503 punch from 0x12eba74 to 0x12fb3bb, (0xf947 bytes) 2024-09-14T22:53:14.222 INFO:teuthology.orchestra.run.smithi076.stdout:504 read 0x4d6dbb1 thru 0x4d6e28f (0x6df bytes) 2024-09-14T22:53:14.222 INFO:teuthology.orchestra.run.smithi076.stdout:507 write 0x1bebc57 thru 0x1bec797 (0xb41 bytes) 2024-09-14T22:53:14.239 INFO:teuthology.orchestra.run.smithi076.stdout:508 write 0xc5cacdf thru 0xc5cd143 (0x2465 bytes) 2024-09-14T22:53:14.242 INFO:teuthology.orchestra.run.smithi076.stdout:509 clone 3 order 21 su 131072 sc 7 2024-09-14T22:53:14.268 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:14.267+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:14.278 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:14.278 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:14.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:14.278 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:14.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.278 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:12 3.8 3:15b9a59a:::rbd_header.11352933567d:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T22:53:14.278 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.778409+0000", 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.4972519750000002, 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0013616660000000001, 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 12 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778409+0000", 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778409+0000", 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778411+0000", 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4139999999999999e-06 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778417+0000", 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8839999999999999e-06 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778425+0000", 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5869999999999993e-06 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778432+0000", 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6869999999999997e-06 2024-09-14T22:53:14.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778491+0000", 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9459000000000001e-05 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778643+0000", 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00015202799999999999 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.778873+0000", 2024-09-14T22:53:14.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230211 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779467+0000", 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00059361799999999997 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779703+0000", 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023638000000000001 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779739+0000", 2024-09-14T22:53:14.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5800000000000003e-05 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.779771+0000", 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1597999999999998e-05 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:25 3.8 3:15b9a59a:::rbd_header.11352933567d:head [watch watch cookie 140460060339536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:08.788248+0000", 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.4874130379999997, 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001194068, 2024-09-14T22:53:14.284 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 25 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788248+0000", 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788248+0000", 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788249+0000", 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6300000000000005e-07 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788253+0000", 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9779999999999999e-06 2024-09-14T22:53:14.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788261+0000", 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6810000000000004e-06 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788266+0000", 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2740000000000001e-06 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788305+0000", 2024-09-14T22:53:14.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9694000000000003e-05 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788354+0000", 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8702999999999998e-05 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.788451+0000", 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7114000000000006e-05 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.789124+0000", 2024-09-14T22:53:14.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067285000000000001 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.789360+0000", 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000236126 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.789395+0000", 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4725999999999999e-05 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:08.789442+0000", 2024-09-14T22:53:14.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7259000000000003e-05 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:966 3.8 3:15b9a59a:::rbd_header.11352933567d:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.733321+0000", 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.5423402909999999, 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058719039999999998, 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 966 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.733321+0000", 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.733321+0000", 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.733323+0000", 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4210000000000002e-06 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.733334+0000", 2024-09-14T22:53:14.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0968000000000001e-05 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.733357+0000", 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2484e-05 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.733367+0000", 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9499999999999996e-06 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.733429+0000", 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2298999999999994e-05 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.733434+0000", 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3709999999999999e-06 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.735737+0000", 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0023023330000000002 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.735832+0000", 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5444000000000003e-05 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.736239+0000", 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00040671099999999998 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.739048+0000", 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0028087519999999999 2024-09-14T22:53:14.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.739091+0000", 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3600999999999998e-05 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.739143+0000", 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2160999999999997e-05 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.739193+0000", 2024-09-14T22:53:14.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9409000000000001e-05 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:980 3.8 3:15b9a59a:::rbd_header.11352933567d:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.752517+0000", 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.523144432, 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012464220000000001, 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.297 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 980 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752517+0000", 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752517+0000", 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752517+0000", 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1600000000000001e-07 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752521+0000", 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8879999999999999e-06 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752530+0000", 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.191e-06 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752535+0000", 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.699e-06 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752567+0000", 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2023000000000002e-05 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752610+0000", 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2453000000000002e-05 2024-09-14T22:53:14.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.752740+0000", 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013045999999999999 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.753275+0000", 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00053456800000000002 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.753663+0000", 2024-09-14T22:53:14.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000388396 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.753721+0000", 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8356999999999997e-05 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.753763+0000", 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1671000000000001e-05 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1001 3.2 3:4550e1e5:::rbd_id.image_client.0-clone1:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.767006+0000", 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.508655522, 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012918820000000001, 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1001 2024-09-14T22:53:14.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767006+0000", 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767006+0000", 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767007+0000", 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.198e-06 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767014+0000", 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9e-06 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767028+0000", 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4297000000000001e-05 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767034+0000", 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5539999999999997e-06 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767091+0000", 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7598e-05 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767225+0000", 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013416099999999999 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 5", 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.767425+0000", 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019970200000000001 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.768151+0000", 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00072596300000000004 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.768224+0000", 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3176999999999994e-05 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.768257+0000", 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3046000000000002e-05 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.768298+0000", 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0286e-05 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1006 3.2 3:4550e1e5:::rbd_id.image_client.0-clone1:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.774542+0000", 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.501119034, 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010594059999999999, 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1006 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.774542+0000", 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.774542+0000", 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.774543+0000", 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7599999999999996e-07 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.774547+0000", 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6430000000000001e-06 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.774555+0000", 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0760000000000003e-06 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.774559+0000", 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3259999999999997e-06 2024-09-14T22:53:14.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.774589+0000", 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0516999999999999e-05 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.774629+0000", 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9548999999999997e-05 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.775602+0000", 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00097251900000000003 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1116 3.8 3:107554a9:::rbd_data.1135de237b71.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:12.986946+0000", 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.288715018, 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012604230000000001, 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.313 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1116 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.986946+0000", 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.986946+0000", 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.986948+0000", 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.767e-06 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.986956+0000", 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2630000000000003e-06 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.986972+0000", 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6296000000000001e-05 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.986981+0000", 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4120000000000001e-06 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987038+0000", 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6879000000000002e-05 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987151+0000", 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000113446 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.987270+0000", 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00011924700000000001 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.988017+0000", 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00074686000000000002 2024-09-14T22:53:14.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.988105+0000", 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7410000000000005e-05 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.988164+0000", 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9252000000000001e-05 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:12.988207+0000", 2024-09-14T22:53:14.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2590999999999999e-05 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1154 3.8 3:107554a9:::rbd_data.1135de237b71.0000000000000067:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.039819+0000", 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.2358421550000001, 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0014902660000000001, 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.319 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1154 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039819+0000", 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039819+0000", 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039821+0000", 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0099999999999998e-06 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039828+0000", 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3730000000000004e-06 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039845+0000", 2024-09-14T22:53:14.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7007000000000002e-05 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039854+0000", 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0019999999999995e-06 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039926+0000", 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1254999999999995e-05 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040016+0000", 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0212999999999998e-05 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040135+0000", 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118892 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040782+0000", 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000646924 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041255+0000", 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000473482 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041284+0000", 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8384000000000001e-05 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041309+0000", 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5724e-05 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.324 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1155 3.8 3:13892000:::rbd_data.1135de237b71.0000000000000068:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.039878+0000", 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.2357835349999999, 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0015375339999999999, 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1155 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039878+0000", 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039878+0000", 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039878+0000", 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039882+0000", 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.45e-06 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039895+0000", 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3709999999999999e-05 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.039903+0000", 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0649999999999999e-06 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040405+0000", 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050169500000000003 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040478+0000", 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3263999999999995e-05 2024-09-14T22:53:14.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.040569+0000", 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1014000000000006e-05 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041045+0000", 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00047524600000000002 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041351+0000", 2024-09-14T22:53:14.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030639399999999998 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041379+0000", 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7515e-05 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.041415+0000", 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6668000000000002e-05 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.330 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1161 3.2 3:443dbe1d:::rbd_data.1135de237b71.0000000000000096:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.042273+0000", 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.233388613, 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056893250000000003, 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1161 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.331 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042273+0000", 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042273+0000", 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042273+0000", 2024-09-14T22:53:14.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042279+0000", 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1760000000000001e-06 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042289+0000", 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9720000000000004e-06 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042297+0000", 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3480000000000004e-06 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042338+0000", 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0810999999999999e-05 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042381+0000", 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3275999999999997e-05 2024-09-14T22:53:14.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 5", 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042436+0000", 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4709000000000001e-05 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.042789+0000", 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00035355800000000002 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.047884+0000", 2024-09-14T22:53:14.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050951319999999996 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.047923+0000", 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8890999999999997e-05 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.047962+0000", 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8569000000000003e-05 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1184 3.18 3:1e624867:::rbd_data.1135de237b71.0000000000000022:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.060704+0000", 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.214956718, 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001798968, 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1184 2024-09-14T22:53:14.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060704+0000", 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060704+0000", 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060706+0000", 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.823e-06 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060716+0000", 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2399999999999996e-06 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060732+0000", 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6566999999999999e-05 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060742+0000", 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.431e-06 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060816+0000", 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4364999999999997e-05 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.060961+0000", 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014507999999999999 2024-09-14T22:53:14.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061151+0000", 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019006399999999999 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061832+0000", 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00068127499999999996 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062421+0000", 2024-09-14T22:53:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058907700000000003 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062467+0000", 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5148000000000002e-05 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062503+0000", 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6897999999999998e-05 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1190 3.5 e29/24 3:a15a6aa5:::rbd_data.1135de237b71.0000000000000028:head v 29'2, mlcod=0'0)", 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.062056+0000", 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.213604771, 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0023360289999999999, 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062056+0000", 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062056+0000", 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062058+0000", 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1200000000000001e-06 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062066+0000", 2024-09-14T22:53:14.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2400000000000007e-06 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062078+0000", 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2714999999999999e-05 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062086+0000", 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062141+0000", 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5590999999999997e-05 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062165+0000", 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3615999999999999e-05 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.064350+0000", 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021847979999999999 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.064392+0000", 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2629999999999997e-05 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1189 3.15 e29/24 3:ac07c023:::rbd_data.1135de237b71.0000000000000027:head v 29'1, mlcod=0'0)", 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.062201+0000", 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.213460612, 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00251272, 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.347 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062201+0000", 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062201+0000", 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062201+0000", 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4999999999999998e-07 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062205+0000", 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9079999999999997e-06 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062211+0000", 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2049999999999999e-06 2024-09-14T22:53:14.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062216+0000", 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3589999999999999e-06 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062259+0000", 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2790000000000002e-05 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062278+0000", 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8947e-05 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.064671+0000", 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0023928019999999999 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.064713+0000", 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2358999999999999e-05 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.351 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1217 3.15 e29/24 3:ac07c023:::rbd_data.1135de237b71.0000000000000027:head v 29'2, mlcod=0'0)", 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.092165+0000", 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.1834966609999999, 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001263842, 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.092165+0000", 2024-09-14T22:53:14.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.092165+0000", 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.092166+0000", 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.646e-06 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.092183+0000", 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6793e-05 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.092199+0000", 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6002e-05 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.092207+0000", 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.507e-06 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.092264+0000", 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.643e-05 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.092282+0000", 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8141000000000001e-05 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.093395+0000", 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011127369999999999 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.093428+0000", 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3586e-05 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1244 3.2 3:40ec7a08:::rbd_data.1135de237b71.0000000000000051:head [set-alloc-hint object_size 1048576 write_size 1048576,write 796163~6871] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.128698+0000", 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.1469628350000001, 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001201686, 2024-09-14T22:53:14.356 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1244 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128698+0000", 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128698+0000", 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128701+0000", 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6840000000000001e-06 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128715+0000", 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4049000000000001e-05 2024-09-14T22:53:14.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128735+0000", 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0276999999999999e-05 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128746+0000", 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0786e-05 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128812+0000", 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5617000000000006e-05 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128904+0000", 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2471000000000003e-05 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 5", 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.128991+0000", 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7031000000000001e-05 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.129564+0000", 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00057282299999999997 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.129843+0000", 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00027882900000000002 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.129875+0000", 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2112e-05 2024-09-14T22:53:14.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.129900+0000", 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5007e-05 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1267 3.1e 3:7c710ef1:::rbd_data.1135de237b71.00000000000000cf:head [set-alloc-hint object_size 1048576 write_size 1048576,write 791591~38008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.153184+0000", 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.122476939, 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001543855, 2024-09-14T22:53:14.362 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1267 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153184+0000", 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153184+0000", 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153186+0000", 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0770000000000002e-06 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153229+0000", 2024-09-14T22:53:14.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2806000000000003e-05 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153249+0000", 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9989000000000001e-05 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153259+0000", 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0176e-05 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153320+0000", 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0492999999999998e-05 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153406+0000", 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6568000000000002e-05 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 6", 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.153499+0000", 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.154207+0000", 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00070765900000000002 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.154672+0000", 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046522799999999998 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.154702+0000", 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0031000000000001e-05 2024-09-14T22:53:14.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.154728+0000", 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5851e-05 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1269 3.2 3:40ec7a08:::rbd_data.1135de237b71.0000000000000051:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.167331+0000", 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.10833051, 2024-09-14T22:53:14.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0019249740000000001, 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1269 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167331+0000", 2024-09-14T22:53:14.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167331+0000", 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167333+0000", 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4260000000000002e-06 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167341+0000", 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7249999999999997e-06 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167367+0000", 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5882000000000001e-05 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167379+0000", 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1804000000000001e-05 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167444+0000", 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5723999999999996e-05 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167504+0000", 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9450999999999997e-05 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 5", 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167597+0000", 2024-09-14T22:53:14.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3474000000000007e-05 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169095+0000", 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001497663 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169188+0000", 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3554000000000006e-05 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169222+0000", 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4038999999999997e-05 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169256+0000", 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3232000000000003e-05 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1277 3.1e 3:7c710ef1:::rbd_data.1135de237b71.00000000000000cf:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.374 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.167412+0000", 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.1082495969999999, 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001844062, 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1277 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167412+0000", 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167412+0000", 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167412+0000", 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8899999999999999e-07 2024-09-14T22:53:14.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167416+0000", 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7050000000000001e-06 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167424+0000", 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3590000000000008e-06 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167431+0000", 2024-09-14T22:53:14.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.268e-06 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167492+0000", 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1230999999999997e-05 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167558+0000", 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5709000000000004e-05 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 6", 2024-09-14T22:53:14.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.167619+0000", 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1811999999999997e-05 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.168500+0000", 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000880955 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169170+0000", 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00066951100000000002 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169212+0000", 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1993999999999998e-05 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.169256+0000", 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3829000000000003e-05 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.380 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1325 3.8 3:15b9a59a:::rbd_header.11352933567d:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.802411+0000", 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.47324969, 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0018956240000000001, 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1325 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.802411+0000", 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.802411+0000", 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.802414+0000", 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6630000000000002e-06 2024-09-14T22:53:14.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.802429+0000", 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4426999999999999e-05 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.802462+0000", 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2922000000000001e-05 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.802477+0000", 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5196e-05 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.802564+0000", 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7460000000000006e-05 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.802568+0000", 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.98e-06 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.804027+0000", 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0014587529999999999 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.804109+0000", 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1681999999999995e-05 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.804307+0000", 2024-09-14T22:53:14.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000198541 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:1480 3.8 3:15b9a59a:::rbd_header.11352933567d:head [watch unwatch cookie 140459993680208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:14.023789+0000", 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.25187251599999999, 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017538, 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:14.386 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 1480 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.023789+0000", 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:14.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.023789+0000", 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.023791+0000", 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.571e-06 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.023803+0000", 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1284000000000001e-05 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.023823+0000", 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0764999999999999e-05 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.023842+0000", 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8468e-05 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.023899+0000", 2024-09-14T22:53:14.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7671999999999999e-05 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.023964+0000", 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5024999999999995e-05 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.024060+0000", 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5786999999999996e-05 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:53:14.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.024687+0000", 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00062720099999999997 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.025416+0000", 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00072818500000000001 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.025469+0000", 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3263999999999997e-05 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:14.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:14.025542+0000", 2024-09-14T22:53:14.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3578000000000006e-05 2024-09-14T22:53:14.392 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.392 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.392 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.392 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:14.392 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:14.392 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:14.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:14.394 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:14.392+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:14.494 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:14.493+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:14.499 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:14.499 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:14.500 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:14.508 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:14.595 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:14.594+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:14.637 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:14.650 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:14.650 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:14.651 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:14.651 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:14.659 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:14.700 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:14.699+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:14.749 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:53:14.757 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:14.757 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:14.778 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:14.778 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:14.796 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:14.795+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:14.802 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:14.802 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:08.764665+0000", 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.0420837699999996, 2024-09-14T22:53:14.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003492059, 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 9 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764665+0000", 2024-09-14T22:53:14.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764665+0000", 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764667+0000", 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.186e-06 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764673+0000", 2024-09-14T22:53:14.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0889999999999996e-06 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764684+0000", 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0944999999999999e-05 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764693+0000", 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3549999999999994e-06 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764739+0000", 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5757000000000003e-05 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764797+0000", 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8242000000000002e-05 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.764879+0000", 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1720999999999999e-05 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.765850+0000", 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000971319 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.768095+0000", 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022448020000000002 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.768130+0000", 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5562999999999998e-05 2024-09-14T22:53:14.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.768157+0000", 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7080000000000002e-05 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:10 3.1c e27/24 3:3831950c:::rbd_directory:head v 27'1, mlcod=0'0)", 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:08.771122+0000", 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.0356270629999997, 2024-09-14T22:53:14.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00304893, 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.771122+0000", 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.771122+0000", 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.771123+0000", 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8200000000000001e-07 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.771128+0000", 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2259999999999996e-06 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.771137+0000", 2024-09-14T22:53:14.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.901e-06 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.771148+0000", 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0869e-05 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.771201+0000", 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3919000000000003e-05 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.771226+0000", 2024-09-14T22:53:14.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4522e-05 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.774138+0000", 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029118030000000001 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.774171+0000", 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3308e-05 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1019 3.a e29/24 3:535385f0:::rbd_children:head v 29'3, mlcod=28'1)", 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:12.783125+0000", 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.0236239629999999, 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029463219999999999, 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.783125+0000", 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.783125+0000", 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.783125+0000", 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5899999999999998e-07 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.783130+0000", 2024-09-14T22:53:14.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2540000000000003e-06 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.783140+0000", 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475999999999999e-05 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.783147+0000", 2024-09-14T22:53:14.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6669999999999999e-06 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.783194+0000", 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6869000000000001e-05 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.783213+0000", 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9644e-05 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.786043+0000", 2024-09-14T22:53:14.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028296800000000002 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.786072+0000", 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8473e-05 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1112 3.1a 3:5a42f79b:::rbd_data.1135de237b71.0000000000000063:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:12.983498+0000", 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.8232507579999999, 2024-09-14T22:53:14.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020052490000000002, 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1112 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983498+0000", 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983498+0000", 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983500+0000", 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.232e-06 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983505+0000", 2024-09-14T22:53:14.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2499999999999997e-06 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983515+0000", 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0526999999999999e-05 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983521+0000", 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0540000000000002e-06 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983576+0000", 2024-09-14T22:53:14.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.507e-05 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983704+0000", 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012800600000000001 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.983844+0000", 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014074199999999999 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.984660+0000", 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00081569900000000002 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.985431+0000", 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00077147000000000001 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.985477+0000", 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5948000000000001e-05 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.985504+0000", 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6251e-05 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1118 3.f e29/24 3:f3dcf4f1:::rbd_data.1135de237b71.0000000000000069:head v 29'8, mlcod=28'6)", 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:12.992038+0000", 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.8147114869999998, 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020445580000000001, 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.830 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992038+0000", 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992038+0000", 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992038+0000", 2024-09-14T22:53:14.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3e-07 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992043+0000", 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2919999999999997e-06 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992052+0000", 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5359999999999998e-06 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992057+0000", 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6589999999999998e-06 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992101+0000", 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4320000000000003e-05 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992120+0000", 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8689999999999999e-05 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.994050+0000", 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019300210000000001 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.994082+0000", 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2410000000000003e-05 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1119 3.1 e29/24 3:81ec2e82:::rbd_data.1135de237b71.000000000000006a:head v 29'1, mlcod=0'0)", 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:12.992115+0000", 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.8146338869999998, 2024-09-14T22:53:14.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002267991, 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992115+0000", 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992115+0000", 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992116+0000", 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4799999999999997e-07 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992122+0000", 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7860000000000002e-06 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992133+0000", 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1342000000000001e-05 2024-09-14T22:53:14.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992140+0000", 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992184+0000", 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3724999999999999e-05 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.992201+0000", 2024-09-14T22:53:14.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7447000000000001e-05 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.994357+0000", 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002155807 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:12.994383+0000", 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6465000000000002e-05 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1150 3.1a 3:5a42f79b:::rbd_data.1135de237b71.0000000000000063:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.039813+0000", 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.766936474, 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024157900000000001, 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1150 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.839 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039813+0000", 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039813+0000", 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039814+0000", 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.268e-06 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039819+0000", 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9869999999999999e-06 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039830+0000", 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1343e-05 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039837+0000", 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039904+0000", 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.711e-05 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039984+0000", 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9457999999999997e-05 2024-09-14T22:53:14.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040081+0000", 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6596999999999997e-05 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.041062+0000", 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098164499999999996 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042140+0000", 2024-09-14T22:53:14.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010776029999999999 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042186+0000", 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6455000000000002e-05 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042229+0000", 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2261000000000001e-05 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1149 3.13 e29/24 3:ca91a77c:::rbd_data.1135de237b71.0000000000000062:head v 29'23, mlcod=28'21)", 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040344+0000", 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.766404753, 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046986110000000001, 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040344+0000", 2024-09-14T22:53:14.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040344+0000", 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040345+0000", 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040352+0000", 2024-09-14T22:53:14.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2140000000000001e-06 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040362+0000", 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0791999999999999e-05 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040369+0000", 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0400000000000004e-06 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040426+0000", 2024-09-14T22:53:14.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6125999999999998e-05 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040445+0000", 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9409e-05 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044990+0000", 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045446159999999996 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045043+0000", 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.346e-05 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1157 3.1 e29/24 3:81ec2e82:::rbd_data.1135de237b71.000000000000006a:head v 29'2, mlcod=0'0)", 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040404+0000", 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7663455260000001, 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004635765, 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.849 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040404+0000", 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040404+0000", 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040408+0000", 2024-09-14T22:53:14.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0620000000000002e-06 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040422+0000", 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4669e-05 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040432+0000", 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1390000000000001e-06 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040438+0000", 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6170000000000003e-06 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040485+0000", 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6961e-05 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040503+0000", 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7611000000000001e-05 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044988+0000", 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044854420000000001 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045039+0000", 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1264000000000002e-05 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1152 3.11 e29/24 3:8a62f64f:::rbd_data.1135de237b71.0000000000000065:head v 29'4, mlcod=28'2)", 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040560+0000", 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7661892369999999, 2024-09-14T22:53:14.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045065640000000002, 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040560+0000", 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040560+0000", 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040560+0000", 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.39e-07 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040565+0000", 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9710000000000003e-06 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040574+0000", 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5869999999999993e-06 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040587+0000", 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2636999999999999e-05 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040622+0000", 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5231999999999997e-05 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040641+0000", 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8822999999999999e-05 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045041+0000", 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044007650000000001 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045067+0000", 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5109999999999998e-05 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1156 3.f e29/24 3:f3dcf4f1:::rbd_data.1135de237b71.0000000000000069:head v 29'9, mlcod=29'7)", 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040601+0000", 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.766148313, 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043627340000000001, 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040601+0000", 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040601+0000", 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040601+0000", 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.41e-07 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040609+0000", 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7279999999999995e-06 2024-09-14T22:53:14.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040614+0000", 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040617+0000", 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0580000000000002e-06 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040654+0000", 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7460999999999999e-05 2024-09-14T22:53:14.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040667+0000", 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2473000000000001e-05 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044906+0000", 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042395790000000003 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044964+0000", 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7157999999999997e-05 2024-09-14T22:53:14.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1183 3.14 3:2876bd93:::rbd_data.1135de237b71.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.059345+0000", 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7474044769999999, 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020004480000000002, 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1183 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059345+0000", 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059345+0000", 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059346+0000", 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2410000000000001e-06 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059352+0000", 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6799999999999998e-06 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059366+0000", 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4633e-05 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059382+0000", 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5543999999999999e-05 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059435+0000", 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3582000000000003e-05 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059556+0000", 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120396 2024-09-14T22:53:14.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.059761+0000", 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020568600000000001 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060655+0000", 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089328499999999996 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061291+0000", 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00063593099999999995 2024-09-14T22:53:14.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061321+0000", 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0341e-05 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061345+0000", 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4128999999999999e-05 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1185 3.14 3:2b715b28:::rbd_data.1135de237b71.0000000000000023:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.060817+0000", 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7459320790000001, 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035081230000000001, 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1185 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.868 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060817+0000", 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060817+0000", 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060818+0000", 2024-09-14T22:53:14.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.026e-06 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060824+0000", 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.485e-06 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060837+0000", 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3585e-05 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060844+0000", 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.255e-06 2024-09-14T22:53:14.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060905+0000", 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0364e-05 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061003+0000", 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8504000000000004e-05 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:53:14.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061119+0000", 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000115276 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064066+0000", 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029471860000000001 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064239+0000", 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017307299999999999 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064286+0000", 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6900000000000002e-05 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064325+0000", 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9468999999999998e-05 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1188 3.1b e29/24 3:dafebb96:::rbd_data.1135de237b71.0000000000000026:head v 29'9, mlcod=28'7)", 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.061772+0000", 2024-09-14T22:53:14.873 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7449767029999999, 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0025913490000000002, 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061772+0000", 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061772+0000", 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061773+0000", 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3e-07 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061780+0000", 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8210000000000004e-06 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061792+0000", 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1836e-05 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061798+0000", 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3799999999999999e-06 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061859+0000", 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0770000000000003e-05 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061879+0000", 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9928999999999998e-05 2024-09-14T22:53:14.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064327+0000", 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024480489999999999 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064364+0000", 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6834e-05 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.877 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1191 3.1b e29/24 3:d8b4be97:::rbd_data.1135de237b71.0000000000000029:head v 29'10, mlcod=28'7)", 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.062301+0000", 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7444479049999999, 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020897860000000002, 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062301+0000", 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062301+0000", 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062302+0000", 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4899999999999995e-07 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062306+0000", 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5539999999999999e-06 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062316+0000", 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5889999999999992e-06 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062321+0000", 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5580000000000002e-06 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062357+0000", 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6287e-05 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062369+0000", 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2502e-05 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064366+0000", 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019970320000000001 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064391+0000", 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4715e-05 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1214 3.3 3:c51feaef:::rbd_data.1135de237b71.000000000000005c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 45082~20454] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.084807+0000", 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7219418630000001, 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031311310000000001, 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1214 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.084807+0000", 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.084807+0000", 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.084808+0000", 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0699999999999996e-07 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.084857+0000", 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8405999999999997e-05 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.084867+0000", 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0084e-05 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.084873+0000", 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7689999999999999e-06 2024-09-14T22:53:14.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.084911+0000", 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7489e-05 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.084998+0000", 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7267000000000003e-05 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085074+0000", 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5610000000000003e-05 2024-09-14T22:53:14.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085971+0000", 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089716500000000001 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.087845+0000", 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0018738209999999999 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.087898+0000", 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3117e-05 2024-09-14T22:53:14.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.087938+0000", 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0596000000000002e-05 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1215 3.3 3:c5621b4f:::rbd_data.1135de237b71.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write 0~19785] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.085233+0000", 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.721516359, 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056322339999999999, 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.887 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1215 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085233+0000", 2024-09-14T22:53:14.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085233+0000", 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085233+0000", 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0200000000000002e-07 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085253+0000", 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9644999999999999e-05 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085268+0000", 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5048999999999999e-05 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085273+0000", 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0989999999999998e-06 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085407+0000", 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013345300000000001 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085528+0000", 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121137 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085656+0000", 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012804799999999999 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.086656+0000", 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010000040000000001 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.090777+0000", 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041215069999999999 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.090829+0000", 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1286999999999999e-05 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.090865+0000", 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6402999999999997e-05 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1270 3.3 3:c51feaef:::rbd_data.1135de237b71.000000000000005c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.167463+0000", 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.63928666, 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050213050000000002, 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.893 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1270 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167463+0000", 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167463+0000", 2024-09-14T22:53:14.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167481+0000", 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8547999999999999e-05 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167487+0000", 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167502+0000", 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4603999999999999e-05 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167509+0000", 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6569999999999997e-06 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167562+0000", 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2432999999999998e-05 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167633+0000", 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1366999999999996e-05 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167733+0000", 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9386e-05 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168410+0000", 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00067717799999999996 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172428+0000", 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040179769999999998 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172458+0000", 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0434999999999999e-05 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172484+0000", 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5694000000000002e-05 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.898 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1271 3.3 3:c5621b4f:::rbd_data.1135de237b71.000000000000005d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.167527+0000", 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6392220609999999, 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050568189999999997, 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.899 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1271 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167527+0000", 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167527+0000", 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167528+0000", 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4499999999999997e-07 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167530+0000", 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6309999999999999e-06 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167538+0000", 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1930000000000004e-06 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167543+0000", 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167987+0000", 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044392999999999999 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168062+0000", 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5197000000000005e-05 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168189+0000", 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126647 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168979+0000", 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00078982100000000001 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172527+0000", 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035487029999999998 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172558+0000", 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0448000000000001e-05 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172584+0000", 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6083000000000001e-05 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1274 3.1a 3:5dc85408:::rbd_data.1135de237b71.000000000000008e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:14.904 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.167553+0000", 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6391966249999999, 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020080340000000001, 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1274 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167553+0000", 2024-09-14T22:53:14.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167553+0000", 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167553+0000", 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7300000000000001e-07 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167563+0000", 2024-09-14T22:53:14.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0431000000000001e-05 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167571+0000", 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0439999999999998e-06 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167576+0000", 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1019999999999996e-06 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167622+0000", 2024-09-14T22:53:14.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6637000000000001e-05 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167699+0000", 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6338e-05 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167788+0000", 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9720000000000002e-05 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:14.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168907+0000", 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00111871 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.169502+0000", 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000595238 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.169532+0000", 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0119000000000001e-05 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:14.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.169561+0000", 2024-09-14T22:53:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8221999999999998e-05 2024-09-14T22:53:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:14.910 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:14.910 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:14.911 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:14.911 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:14.911 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone2 from 0xc5cd144 (overlap 0x45c6e79) to 0x0 2024-09-14T22:53:14.912 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:14.912 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:14.912 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:14.913 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:14.913 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:14.913 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:14.914 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:14.914 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:14.910+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:14.923 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:14.924 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:15.012 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:15.010+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:15.084 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:15.085 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:15.094 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:15.112 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:15.111+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:15.143 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T22:53:15.213 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:15.212+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:15.239 INFO:teuthology.orchestra.run.smithi076.stdout:510 write 0x1036044 thru 0x10378e4 (0x18a1 bytes) 2024-09-14T22:53:15.241 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:15.241 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:15.247 INFO:teuthology.orchestra.run.smithi076.stdout:512 write 0x90fc6ca thru 0x9101f25 (0x585c bytes) 2024-09-14T22:53:15.251 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:15.255 INFO:teuthology.orchestra.run.smithi076.stdout:513 read 0x6d3d414 thru 0x6d4283e (0x542b bytes) 2024-09-14T22:53:15.255 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:15.256 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:15.256 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:15.257 INFO:teuthology.orchestra.run.smithi076.stdout:517 trunc from 0xc5cd144 to 0xddc134f 2024-09-14T22:53:15.266 INFO:teuthology.orchestra.run.smithi076.stdout:518 read 0xcc1a9a2 thru 0xcc296c2 (0xed21 bytes) 2024-09-14T22:53:15.269 INFO:teuthology.orchestra.run.smithi076.stdout:520 trunc from 0xddc134f to 0xbca809d 2024-09-14T22:53:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:521 punch from 0x703541 to 0x712bc1, (0xf680 bytes) 2024-09-14T22:53:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:522 punch from 0x7af73ae to 0x7b03443, (0xc095 bytes) 2024-09-14T22:53:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:523 read 0xb1d9dc6 thru 0xb1e7697 (0xd8d2 bytes) 2024-09-14T22:53:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:524 punch from 0x32f0535 to 0x32f5dc1, (0x588c bytes) 2024-09-14T22:53:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:525 punch from 0x9e593f0 to 0x9e5ab09, (0x1719 bytes) 2024-09-14T22:53:15.290 INFO:teuthology.orchestra.run.smithi076.stdout:526 write 0xa96577f thru 0xa968489 (0x2d0b bytes) 2024-09-14T22:53:15.292 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:15.295 INFO:teuthology.orchestra.run.smithi076.stdout:529 trunc from 0xbca809d to 0xdc9a496 2024-09-14T22:53:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:530 read 0x3cb63cd thru 0x3cbedb3 (0x89e7 bytes) 2024-09-14T22:53:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:531 read 0xce0c0c thru 0xce5002 (0x43f7 bytes) 2024-09-14T22:53:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:534 punch from 0x308c7a1 to 0x3096a1b, (0xa27a bytes) 2024-09-14T22:53:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:535 read 0xa1ff17b thru 0xa20c848 (0xd6ce bytes) 2024-09-14T22:53:15.301 INFO:teuthology.orchestra.run.smithi076.stdout:536 read 0x1a9cbac thru 0x1aa7198 (0xa5ed bytes) 2024-09-14T22:53:15.302 INFO:teuthology.orchestra.run.smithi076.stdout:537 read 0x2d2e17a thru 0x2d3222b (0x40b2 bytes) 2024-09-14T22:53:15.302 INFO:teuthology.orchestra.run.smithi076.stdout:539 punch from 0xe67f31 to 0xe6b29c, (0x336b bytes) 2024-09-14T22:53:15.314 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:15.313+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:15.315 INFO:teuthology.orchestra.run.smithi076.stdout:541 trunc from 0xdc9a496 to 0x22f3fe9 2024-09-14T22:53:15.352 INFO:teuthology.orchestra.run.smithi076.stdout:544 trunc from 0x22f3fe9 to 0x5b1dba1 2024-09-14T22:53:15.359 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:15.366 INFO:teuthology.orchestra.run.smithi076.stdout:547 trunc from 0x5b1dba1 to 0xc033615 2024-09-14T22:53:15.373 INFO:teuthology.orchestra.run.smithi076.stdout:548 trunc from 0xc033615 to 0xaf2cd20 2024-09-14T22:53:15.380 INFO:teuthology.orchestra.run.smithi076.stdout:549 write 0x501a341 thru 0x5022050 (0x7d10 bytes) 2024-09-14T22:53:15.384 INFO:teuthology.orchestra.run.smithi076.stdout:550 write 0xaa0bf9e thru 0xaa168ed (0xa950 bytes) 2024-09-14T22:53:15.387 INFO:teuthology.orchestra.run.smithi076.stdout:551 writesame 0x19aea81 thru 0x19b1bee data_size 0x13(0x316e bytes) 2024-09-14T22:53:15.393 INFO:teuthology.orchestra.run.smithi076.stdout:552 trunc from 0xaf2cd20 to 0xd947110 2024-09-14T22:53:15.399 INFO:teuthology.orchestra.run.smithi076.stdout:553 write 0x65a37fe thru 0x65adb3c (0xa33f bytes) 2024-09-14T22:53:15.404 INFO:teuthology.orchestra.run.smithi076.stdout:554 punch from 0xd59c9b9 to 0xd5a5ff9, (0x9640 bytes) 2024-09-14T22:53:15.407 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:15.407 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:15.411 INFO:teuthology.orchestra.run.smithi076.stdout:556 trunc from 0xd947110 to 0x6c6fc2a 2024-09-14T22:53:15.415 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:15.413+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:15.416 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:15.423 INFO:teuthology.orchestra.run.smithi076.stdout:557 writesame 0x897ddb thru 0x89aef2 data_size 0x4(0x3118 bytes) 2024-09-14T22:53:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:559 read 0x2df1b10 thru 0x2df1ff3 (0x4e4 bytes) 2024-09-14T22:53:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:560 punch from 0x427f760 to 0x4284ef7, (0x5797 bytes) 2024-09-14T22:53:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:561 read 0x1c7ffd3 thru 0x1c8600a (0x6038 bytes) 2024-09-14T22:53:15.429 INFO:teuthology.orchestra.run.smithi076.stdout:562 read 0x4651699 thru 0x4660904 (0xf26c bytes) 2024-09-14T22:53:15.438 INFO:teuthology.orchestra.run.smithi076.stdout:563 write 0x96b881a thru 0x96be4dc (0x5cc3 bytes) 2024-09-14T22:53:15.441 INFO:teuthology.orchestra.run.smithi076.stdout:565 writesame 0x6452552 thru 0x645d7c1 data_size 0x28(0xb270 bytes) 2024-09-14T22:53:15.445 INFO:teuthology.orchestra.run.smithi076.stdout:568 write 0x246c1c9 thru 0x2478e77 (0xccaf bytes) 2024-09-14T22:53:15.453 INFO:teuthology.orchestra.run.smithi076.stdout:571 trunc from 0x96be4dd to 0xea27c2c 2024-09-14T22:53:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:572 punch from 0xadc93b9 to 0xadc9add, (0x724 bytes) 2024-09-14T22:53:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:574 punch from 0x1d62ddc to 0x1d6b593, (0x87b7 bytes) 2024-09-14T22:53:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:575 read 0xe738872 thru 0xe746122 (0xd8b1 bytes) 2024-09-14T22:53:15.458 INFO:teuthology.orchestra.run.smithi076.stdout:577 write 0xd4bb620 thru 0xd4bc54c (0xf2d bytes) 2024-09-14T22:53:15.461 INFO:teuthology.orchestra.run.smithi076.stdout:578 trunc from 0xea27c2c to 0xeb79b5c 2024-09-14T22:53:15.468 INFO:teuthology.orchestra.run.smithi076.stdout:579 write 0x794417b thru 0x794ea52 (0xa8d8 bytes) 2024-09-14T22:53:15.471 INFO:teuthology.orchestra.run.smithi076.stdout:580 read 0x5855a64 thru 0x585abcd (0x516a bytes) 2024-09-14T22:53:15.472 INFO:teuthology.orchestra.run.smithi076.stdout:581 punch from 0x76059a1 to 0x760d7c9, (0x7e28 bytes) 2024-09-14T22:53:15.472 INFO:teuthology.orchestra.run.smithi076.stdout:582 writesame 0x3817933 thru 0x38250cb data_size 0x1(0xd799 bytes) 2024-09-14T22:53:15.507 INFO:teuthology.orchestra.run.smithi076.stdout:583 read 0x2fd969d thru 0x2fe8571 (0xeed5 bytes) 2024-09-14T22:53:15.507 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:15.508 INFO:teuthology.orchestra.run.smithi076.stdout:585 write 0x3e344d9 thru 0x3e3c3e5 (0x7f0d bytes) 2024-09-14T22:53:15.509 INFO:teuthology.orchestra.run.smithi076.stdout:587 writesame 0x4430c3d thru 0x443db18 data_size 0x12(0xcedc bytes) 2024-09-14T22:53:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:593 read 0x372079a thru 0x3720e1f (0x686 bytes) 2024-09-14T22:53:15.515 INFO:teuthology.orchestra.run.smithi076.stdout:595 clone 4 order 23 su 524288 sc 10 2024-09-14T22:53:15.515 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:15.514+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:15.562 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:15.562 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:15.570 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:15.615 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:15.614+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:15.673 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:15.673 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:15.682 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:15.717 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:15.715+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:15.787 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:53:15.831 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:15.831 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:15.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:15.841 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:15.840+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:15.861 INFO:teuthology.orchestra.run.smithi076.stdout:flattening image image_client.0-clone3 2024-09-14T22:53:15.917 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:15.916+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:15.952 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:15.952 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:15.960 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:16.018 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:16.016+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:16.058 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T22:53:16.059 INFO:teuthology.orchestra.run.smithi076.stdout:596 punch from 0x53f60fb to 0x53f9478, (0x337d bytes) 2024-09-14T22:53:16.062 INFO:teuthology.orchestra.run.smithi076.stdout:597 punch from 0xd8f2091 to 0xd8ffcc6, (0xdc35 bytes) 2024-09-14T22:53:16.063 INFO:teuthology.orchestra.run.smithi076.stdout:598 punch from 0xead590a to 0xeadf637, (0x9d2d bytes) 2024-09-14T22:53:16.063 INFO:teuthology.orchestra.run.smithi076.stdout:599 clone 5 order 19 su 2048 sc 12 2024-09-14T22:53:16.066 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:16.066 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:16.075 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:16.118 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:16.117+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:16.219 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:16.219 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:16.220 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:16.218+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:16.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:16.320 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:16.319+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:16.331 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:16.331 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:16.340 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:16.421 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:16.419+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:16.484 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:16.492 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:16.522 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:16.521+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:16.595 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:16.596 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:16.604 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:16.622 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:16.621+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:16.707 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:16.707 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:16.715 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:16.779 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:16.758+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:16.823 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:16.823 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:16.824 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:16.822+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:16.832 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:16.850 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone4 from 0xeb79b5c (overlap 0xeb79b5c) to 0xbc615e3 2024-09-14T22:53:16.924 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:16.923+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:16.940 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:16.963 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:16.963 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:16.973 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:17.008 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T22:53:17.025 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:17.023+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:17.079 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:17.080 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:17.099 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:17.099 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:17.116 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:17.116 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:17.126 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:17.124+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:17.134 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:17.134 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:17.142 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:17.142 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:17.142 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:17.142 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:17.152 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:17.152 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:17.154 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:17.169 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:17.170 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:17.187 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:17.187 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:17.218 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:17.219 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:17.227 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:17.225+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:17.274 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:17.274 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:17.274 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:17.274 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:17.275 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:17.285 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:17.328 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:17.327+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:17.334 INFO:teuthology.orchestra.run.smithi076.stdout:601 write 0x121b671 thru 0x122720a (0xbb9a bytes) 2024-09-14T22:53:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4232.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.887690062000001, 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010310849999999999, 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8500000000000002e-07 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530823+0000", 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0277e-05 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530835+0000", 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1459e-05 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530845+0000", 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0239e-05 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530907+0000", 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1981000000000002e-05 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530934+0000", 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6701999999999999e-05 2024-09-14T22:53:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.531810+0000", 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00087597399999999996 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.531844+0000", 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4068e-05 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1179 3.4 e29/24 3:226f6f0f:::rbd_data.1135de237b71.000000000000001d:head v 29'5, mlcod=28'3)", 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.053425+0000", 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.36507764, 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00075315400000000002, 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053425+0000", 2024-09-14T22:53:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053425+0000", 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053425+0000", 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1600000000000002e-07 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053434+0000", 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.405e-06 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053459+0000", 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5199e-05 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053471+0000", 2024-09-14T22:53:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2252000000000001e-05 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053527+0000", 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5797000000000001e-05 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.053548+0000", 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0908999999999999e-05 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.054134+0000", 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058633100000000005 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.054178+0000", 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3945e-05 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1186 3.4 e29/24 3:2773f73b:::rbd_data.1135de237b71.0000000000000024:head v 29'6, mlcod=28'4)", 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.061416+0000", 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.3570862650000004, 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00078209299999999998, 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061416+0000", 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061416+0000", 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061418+0000", 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1990000000000001e-06 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061428+0000", 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0706e-05 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061447+0000", 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8989999999999999e-05 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061460+0000", 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3176999999999999e-05 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061598+0000", 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013775699999999999 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.061620+0000", 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1328999999999999e-05 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062159+0000", 2024-09-14T22:53:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00053960300000000004 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.062198+0000", 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9332000000000002e-05 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:1202 3.4 e29/24 3:2292a349:::rbd_data.1135de237b71.000000000000000f:head v 29'7, mlcod=29'5)", 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:13.073403+0000", 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.3450991600000002, 2024-09-14T22:53:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00078782099999999996, 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073403+0000", 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073403+0000", 2024-09-14T22:53:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073405+0000", 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2109999999999999e-06 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073441+0000", 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6032999999999999e-05 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073457+0000", 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6387000000000001e-05 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073465+0000", 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0309999999999998e-06 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073522+0000", 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6371e-05 2024-09-14T22:53:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.073545+0000", 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3428999999999999e-05 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.074162+0000", 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00061683599999999995 2024-09-14T22:53:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:13.074191+0000", 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9522999999999998e-05 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:17.439 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:17.441 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:17.441 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:17.439+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:602 read 0x9675e04 thru 0x9684286 (0xe483 bytes) 2024-09-14T22:53:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:603 write 0x6fd0cb6 thru 0x6fd76e4 (0x6a2f bytes) 2024-09-14T22:53:17.542 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:17.540+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:17.560 INFO:teuthology.orchestra.run.smithi076.stdout:606 write 0x801c365 thru 0x8028c80 (0xc91c bytes) 2024-09-14T22:53:17.580 INFO:teuthology.orchestra.run.smithi076.stdout:607 punch from 0x17d34f5 to 0x17e3360, (0xfe6b bytes) 2024-09-14T22:53:17.580 INFO:teuthology.orchestra.run.smithi076.stdout:608 write 0xc1d3ec8 thru 0xc1dc6ad (0x87e6 bytes) 2024-09-14T22:53:17.625 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:17.625 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:17.625 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:17.625 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:17.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:613 read 0xae6b975 thru 0xae78f85 (0xd611 bytes) 2024-09-14T22:53:17.665 INFO:teuthology.orchestra.run.smithi076.stdout:615 writesame 0x5ee83e2 thru 0x5eeaebc data_size 0x17(0x2adb bytes) 2024-09-14T22:53:17.665 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:17.663+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:17.675 INFO:teuthology.orchestra.run.smithi076.stdout:616 punch from 0x274cf5c to 0x2751ab3, (0x4b57 bytes) 2024-09-14T22:53:17.675 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:17.676 INFO:teuthology.orchestra.run.smithi076.stdout:618 write 0x15c28fd thru 0x15c6ca3 (0x43a7 bytes) 2024-09-14T22:53:17.680 INFO:teuthology.orchestra.run.smithi076.stdout:619 write 0x35fb4d thru 0x3667a5 (0x6c59 bytes) 2024-09-14T22:53:17.743 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:17.742+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:17.772 INFO:teuthology.orchestra.run.smithi076.stdout:620 clone 6 order 23 su 524288 sc 8 2024-09-14T22:53:17.800 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:53:17.801 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:17.801 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:17.812 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:17.844 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:17.843+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:17.863 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone5 from 0xeb79b5c (overlap 0xeb79b5c) to 0x5e30af1 2024-09-14T22:53:17.945 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:17.944+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:18.013 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8145 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 169984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715796+0000", 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29434471899999998, 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.047609619999999998, 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8145 2024-09-14T22:53:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715796+0000", 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715796+0000", 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715797+0000", 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6800000000000002e-07 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715798+0000", 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.398e-06 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715803+0000", 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7700000000000001e-06 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715807+0000", 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2719999999999999e-06 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763198+0000", 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.047390925 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763248+0000", 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9916000000000001e-05 2024-09-14T22:53:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763406+0000", 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000158071 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8147 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 194560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715814+0000", 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.2943267, 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.047841118000000002, 2024-09-14T22:53:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8147 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715814+0000", 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715814+0000", 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715815+0000", 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.27e-07 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715817+0000", 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2819999999999999e-06 2024-09-14T22:53:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715824+0000", 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.472e-06 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715828+0000", 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8380000000000001e-06 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763448+0000", 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.047619939999999999 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763539+0000", 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0996999999999996e-05 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763655+0000", 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116162 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8148 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 219136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715835+0000", 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29430621099999998, 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048054233000000002, 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8148 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715835+0000", 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715835+0000", 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715835+0000", 2024-09-14T22:53:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.58e-07 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715837+0000", 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0600000000000002e-06 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715845+0000", 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2749999999999998e-06 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715856+0000", 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1175999999999999e-05 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763688+0000", 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.047832254999999997 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763771+0000", 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3154999999999999e-05 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763889+0000", 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000117954 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8149 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 243712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715862+0000", 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29427939400000003, 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048176264000000003, 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8149 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715862+0000", 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715862+0000", 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715862+0000", 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715864+0000", 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.001e-06 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715870+0000", 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9429999999999997e-06 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715876+0000", 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6450000000000002e-06 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763925+0000", 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048049103000000003 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.763974+0000", 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8806e-05 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764038+0000", 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4372e-05 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8154 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 268288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715881+0000", 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29426043200000002, 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048304260000000002, 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8154 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715881+0000", 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715881+0000", 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715881+0000", 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.27e-07 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715883+0000", 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6500000000000001e-06 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715888+0000", 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5130000000000004e-06 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715893+0000", 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8600000000000001e-06 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764067+0000", 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048173599999999997 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764123+0000", 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6725999999999999e-05 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764185+0000", 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1583999999999999e-05 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8156 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 292864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715898+0000", 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29424324400000001, 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048421865000000001, 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8156 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715898+0000", 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715898+0000", 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715898+0000", 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2399999999999999e-07 2024-09-14T22:53:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715900+0000", 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.668e-06 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715910+0000", 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0178e-05 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715915+0000", 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764212+0000", 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048297157 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764259+0000", 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6968999999999997e-05 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764320+0000", 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0995999999999997e-05 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8157 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 317440~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715943+0000", 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29419805500000001, 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048629679000000002, 2024-09-14T22:53:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8157 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715943+0000", 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715943+0000", 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715943+0000", 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715945+0000", 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.835e-06 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715951+0000", 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.066e-06 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715956+0000", 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4129999999999999e-06 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764364+0000", 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048407853000000001 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764456+0000", 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2182000000000006e-05 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764573+0000", 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116823 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8158 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 342016~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715961+0000", 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29418002799999998, 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048728501, 2024-09-14T22:53:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8158 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715961+0000", 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715961+0000", 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715961+0000", 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0600000000000001e-07 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715963+0000", 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6700000000000001e-06 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715970+0000", 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6930000000000001e-06 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715974+0000", 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2529999999999998e-06 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764587+0000", 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.04861277 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764629+0000", 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1977000000000001e-05 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764690+0000", 2024-09-14T22:53:18.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0832000000000003e-05 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8159 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 366592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715979+0000", 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29416232799999997, 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048861384000000001, 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.051 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8159 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715979+0000", 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715979+0000", 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715979+0000", 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2199999999999999e-07 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715982+0000", 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.424e-06 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.053 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715987+0000", 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4280000000000004e-06 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715991+0000", 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0049999999999998e-06 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.054 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.055 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764719+0000", 2024-09-14T22:53:18.055 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048728511000000002 2024-09-14T22:53:18.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764768+0000", 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8733000000000002e-05 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764840+0000", 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1860999999999995e-05 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8160 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 391168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.715996+0000", 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29414472699999999, 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048977482000000003, 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8160 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715996+0000", 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715996+0000", 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.058 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715997+0000", 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.715999+0000", 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0530000000000001e-06 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716005+0000", 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9569999999999999e-06 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.059 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716010+0000", 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.276e-06 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764871+0000", 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048860801000000002 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764916+0000", 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5606999999999999e-05 2024-09-14T22:53:18.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764974+0000", 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7450000000000001e-05 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8166 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 415744~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716022+0000", 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29411942800000002, 2024-09-14T22:53:18.061 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049087952999999997, 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8166 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716022+0000", 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716022+0000", 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716022+0000", 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9900000000000001e-07 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716024+0000", 2024-09-14T22:53:18.063 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2290000000000002e-06 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716031+0000", 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716036+0000", 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1769999999999997e-06 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.064 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.764999+0000", 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048963802000000001 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765048+0000", 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8044000000000003e-05 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765110+0000", 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2026000000000006e-05 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.065 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8167 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 440320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716042+0000", 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29409948699999999, 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049183565999999998, 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.066 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8167 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716042+0000", 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716042+0000", 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.067 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716042+0000", 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716077+0000", 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4700999999999999e-05 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716084+0000", 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3989999999999998e-06 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.068 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716089+0000", 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9339999999999997e-06 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765125+0000", 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049036141999999998 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765167+0000", 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2147999999999997e-05 2024-09-14T22:53:18.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765225+0000", 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7890999999999999e-05 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8168 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 464896~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716096+0000", 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29404501100000002, 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049257541000000002, 2024-09-14T22:53:18.070 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8168 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716096+0000", 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.071 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716096+0000", 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716096+0000", 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.072 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716099+0000", 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.21e-06 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716104+0000", 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5639999999999996e-06 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716109+0000", 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3789999999999999e-06 2024-09-14T22:53:18.073 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765250+0000", 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049141365999999999 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765297+0000", 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7314999999999997e-05 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765354+0000", 2024-09-14T22:53:18.074 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6258000000000002e-05 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8169 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 489472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716114+0000", 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29402737800000001, 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049352636999999998, 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.075 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8169 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716114+0000", 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716114+0000", 2024-09-14T22:53:18.076 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716114+0000", 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0199999999999998e-07 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716116+0000", 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7349999999999999e-06 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.077 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716121+0000", 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999997e-06 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716125+0000", 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.189e-06 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765369+0000", 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.04924394 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.078 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765412+0000", 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2873000000000001e-05 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765466+0000", 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4258000000000001e-05 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8170 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 514048~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.079 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716130+0000", 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29401090699999999, 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049469752999999998, 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8170 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.080 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716130+0000", 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716130+0000", 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716130+0000", 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.0400000000000002e-07 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.081 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716132+0000", 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.75e-06 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716138+0000", 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.339e-06 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716142+0000", 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3139999999999997e-06 2024-09-14T22:53:18.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765492+0000", 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049350343999999997 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765539+0000", 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6752999999999998e-05 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765600+0000", 2024-09-14T22:53:18.083 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0949000000000002e-05 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8177 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 538624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716147+0000", 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29399433699999999, 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049566468000000002, 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.084 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8177 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716147+0000", 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.085 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716147+0000", 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716147+0000", 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5799999999999999e-07 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716149+0000", 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7990000000000001e-06 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.086 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716156+0000", 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1300000000000003e-06 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716161+0000", 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5029999999999999e-06 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765615+0000", 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049454649000000003 2024-09-14T22:53:18.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765657+0000", 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1658e-05 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765713+0000", 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6171000000000001e-05 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8178 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 563200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716167+0000", 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29397357800000001, 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049692844999999999, 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8178 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.089 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716167+0000", 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716167+0000", 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716168+0000", 2024-09-14T22:53:18.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716170+0000", 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2450000000000001e-06 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716176+0000", 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.071e-06 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716181+0000", 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5449999999999997e-06 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765739+0000", 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049558075 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765793+0000", 2024-09-14T22:53:18.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3919999999999999e-05 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765860+0000", 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7595000000000006e-05 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8179 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 587776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716187+0000", 2024-09-14T22:53:18.093 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29395442399999999, 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049809995000000003, 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8179 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716187+0000", 2024-09-14T22:53:18.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716187+0000", 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716187+0000", 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716189+0000", 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.888e-06 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716194+0000", 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5289999999999999e-06 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716199+0000", 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.262e-06 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.096 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765892+0000", 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049693647000000001 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765942+0000", 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9920000000000003e-05 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.765997+0000", 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4296999999999999e-05 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.097 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8180 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 612352~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.716204+0000", 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29393748400000003, 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.049924902, 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.098 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8180 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716204+0000", 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.716204+0000", 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717784+0000", 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0015800079999999999 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717788+0000", 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4290000000000003e-06 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717801+0000", 2024-09-14T22:53:18.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3402e-05 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717809+0000", 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2649999999999999e-06 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.766022+0000", 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048213484000000001 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.766068+0000", 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5534000000000001e-05 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.766128+0000", 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0779999999999997e-05 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:8181 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 636928~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.102 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.717821+0000", 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29231961099999998, 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048438237000000002, 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 8181 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717821+0000", 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717821+0000", 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717822+0000", 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1799999999999995e-07 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:18.104 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717827+0000", 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4750000000000004e-06 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717835+0000", 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4889999999999995e-06 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.717841+0000", 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7869999999999998e-06 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.766145+0000", 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.048303947999999999 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.766187+0000", 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2193000000000001e-05 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.766260+0000", 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2726999999999997e-05 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.106 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:18.107 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:18.107 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:18.108 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:18.109 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:18.107+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:18.209 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:18.208+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:18.219 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:18.219 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:18.219 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:18.220 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:18.229 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:18.312 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:18.311+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:18.379 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:18.379 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:18.379 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:18.379 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:18.379 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:18.388 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:18.429 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:18.428+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:18.511 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:18.509+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:18.537 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1149 3.13 e29/24 3:ca91a77c:::rbd_data.1135de237b71.0000000000000062:head v 29'23, mlcod=28'21)", 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040344+0000", 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4950641200000003, 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046986110000000001, 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.538 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040344+0000", 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040344+0000", 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040345+0000", 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.540000000000001e-07 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040352+0000", 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2140000000000001e-06 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040362+0000", 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0791999999999999e-05 2024-09-14T22:53:18.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040369+0000", 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0400000000000004e-06 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040426+0000", 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6125999999999998e-05 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040445+0000", 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9409e-05 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044990+0000", 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045446159999999996 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045043+0000", 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.346e-05 2024-09-14T22:53:18.542 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1157 3.1 e29/24 3:81ec2e82:::rbd_data.1135de237b71.000000000000006a:head v 29'2, mlcod=0'0)", 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040404+0000", 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4950048929999999, 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004635765, 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040404+0000", 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040404+0000", 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040408+0000", 2024-09-14T22:53:18.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0620000000000002e-06 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040422+0000", 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4669e-05 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040432+0000", 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1390000000000001e-06 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040438+0000", 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6170000000000003e-06 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040485+0000", 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6961e-05 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040503+0000", 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7611000000000001e-05 2024-09-14T22:53:18.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044988+0000", 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044854420000000001 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045039+0000", 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1264000000000002e-05 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1152 3.11 e29/24 3:8a62f64f:::rbd_data.1135de237b71.0000000000000065:head v 29'4, mlcod=28'2)", 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040560+0000", 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4948486040000004, 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045065640000000002, 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040560+0000", 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040560+0000", 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040560+0000", 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.39e-07 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040565+0000", 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9710000000000003e-06 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040574+0000", 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5869999999999993e-06 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040587+0000", 2024-09-14T22:53:18.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2636999999999999e-05 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040622+0000", 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5231999999999997e-05 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040641+0000", 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8822999999999999e-05 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045041+0000", 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044007650000000001 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045067+0000", 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5109999999999998e-05 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1156 3.f e29/24 3:f3dcf4f1:::rbd_data.1135de237b71.0000000000000069:head v 29'9, mlcod=29'7)", 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040601+0000", 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4948076800000001, 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043627340000000001, 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040601+0000", 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040601+0000", 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040601+0000", 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.41e-07 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040609+0000", 2024-09-14T22:53:18.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7279999999999995e-06 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040614+0000", 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9359999999999998e-06 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040617+0000", 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0580000000000002e-06 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040654+0000", 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7460999999999999e-05 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040667+0000", 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2473000000000001e-05 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044906+0000", 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042395790000000003 2024-09-14T22:53:18.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044964+0000", 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7157999999999997e-05 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1185 3.14 3:2b715b28:::rbd_data.1135de237b71.0000000000000023:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.060817+0000", 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4745914459999998, 2024-09-14T22:53:18.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035081230000000001, 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1185 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060817+0000", 2024-09-14T22:53:18.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060817+0000", 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060818+0000", 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.026e-06 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060824+0000", 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.485e-06 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060837+0000", 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3585e-05 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060844+0000", 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.255e-06 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.060905+0000", 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0364e-05 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061003+0000", 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8504000000000004e-05 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061119+0000", 2024-09-14T22:53:18.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000115276 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064066+0000", 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029471860000000001 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064239+0000", 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017307299999999999 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064286+0000", 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6900000000000002e-05 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064325+0000", 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9468999999999998e-05 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1215 3.3 3:c5621b4f:::rbd_data.1135de237b71.000000000000005d:head [set-alloc-hint object_size 1048576 write_size 1048576,write 0~19785] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:18.563 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.085233+0000", 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4501757260000003, 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056322339999999999, 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1215 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085233+0000", 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085233+0000", 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085233+0000", 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0200000000000002e-07 2024-09-14T22:53:18.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085253+0000", 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9644999999999999e-05 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085268+0000", 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5048999999999999e-05 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085273+0000", 2024-09-14T22:53:18.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0989999999999998e-06 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085407+0000", 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013345300000000001 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085528+0000", 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121137 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.085656+0000", 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012804799999999999 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.086656+0000", 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010000040000000001 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.090777+0000", 2024-09-14T22:53:18.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041215069999999999 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.090829+0000", 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1286999999999999e-05 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.090865+0000", 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6402999999999997e-05 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1270 3.3 3:c51feaef:::rbd_data.1135de237b71.000000000000005c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.167463+0000", 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.3679460270000003, 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050213050000000002, 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.570 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1270 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167463+0000", 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167463+0000", 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167481+0000", 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8547999999999999e-05 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167487+0000", 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0260000000000004e-06 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167502+0000", 2024-09-14T22:53:18.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4603999999999999e-05 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167509+0000", 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6569999999999997e-06 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167562+0000", 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2432999999999998e-05 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167633+0000", 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1366999999999996e-05 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167733+0000", 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9386e-05 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168410+0000", 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00067717799999999996 2024-09-14T22:53:18.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172428+0000", 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040179769999999998 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172458+0000", 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0434999999999999e-05 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172484+0000", 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5694000000000002e-05 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1271 3.3 3:c5621b4f:::rbd_data.1135de237b71.000000000000005d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.167527+0000", 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.3678814280000005, 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050568189999999997, 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.576 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1271 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167527+0000", 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167527+0000", 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167528+0000", 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4499999999999997e-07 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167530+0000", 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6309999999999999e-06 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167538+0000", 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1930000000000004e-06 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167543+0000", 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6210000000000001e-06 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.167987+0000", 2024-09-14T22:53:18.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044392999999999999 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168062+0000", 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5197000000000005e-05 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168189+0000", 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000126647 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.168979+0000", 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00078982100000000001 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172527+0000", 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035487029999999998 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172558+0000", 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0448000000000001e-05 2024-09-14T22:53:18.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.172584+0000", 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6083000000000001e-05 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:6019 3.1a 3:58496291:::rbd_data.113535cec067.0000000000000025:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 10240~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.402466+0000", 2024-09-14T22:53:18.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.1329425550000001, 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003795016, 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6019 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.402466+0000", 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.402466+0000", 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.402468+0000", 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0420000000000001e-06 2024-09-14T22:53:18.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.403130+0000", 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000661722 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.403141+0000", 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1603000000000001e-05 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.403149+0000", 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3000000000000004e-06 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.403209+0000", 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0668999999999998e-05 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.403348+0000", 2024-09-14T22:53:18.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013904499999999999 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.403696+0000", 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00034773399999999999 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.405591+0000", 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001895065 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.406163+0000", 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057151299999999995 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.406218+0000", 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5748000000000002e-05 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.406261+0000", 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2574999999999998e-05 2024-09-14T22:53:18.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:6750 3.11 e33/24 3:8cb7e912:::rbd_object_map.113535cec067:head v 33'42, mlcod=33'39)", 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.537379+0000", 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.99802973699999997, 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043537510000000003, 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.589 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537379+0000", 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537379+0000", 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537379+0000", 2024-09-14T22:53:18.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3200000000000005e-07 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537384+0000", 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4560000000000002e-06 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537392+0000", 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5320000000000006e-06 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537398+0000", 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5500000000000002e-06 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537445+0000", 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7324000000000003e-05 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537466+0000", 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.073e-05 2024-09-14T22:53:18.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.541703+0000", 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042373410000000004 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.541733+0000", 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9286e-05 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:6752 3.1a 3:5d1e8d6c:::rbd_data.113535cec067.00000000000000d9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 333824~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.537776+0000", 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.99763232599999996, 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038609, 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6752 2024-09-14T22:53:18.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537776+0000", 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537776+0000", 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537778+0000", 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.68e-06 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537783+0000", 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2179999999999998e-06 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537792+0000", 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0119999999999994e-06 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537798+0000", 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.66e-06 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.537855+0000", 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7407000000000001e-05 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538008+0000", 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015289000000000001 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538210+0000", 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000201953 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539178+0000", 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00096834199999999997 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.541548+0000", 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002369641 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.541597+0000", 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8511999999999999e-05 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.541637+0000", 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0584999999999999e-05 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:6751 3.11 e33/24 3:8cb7e912:::rbd_object_map.113535cec067:head v 33'43, mlcod=33'39)", 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.538157+0000", 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.99725154000000005, 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036091909999999999, 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538157+0000", 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538157+0000", 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538158+0000", 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:53:18.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538163+0000", 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.994e-06 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538171+0000", 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0409999999999998e-06 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538178+0000", 2024-09-14T22:53:18.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538224+0000", 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6119999999999999e-05 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.538250+0000", 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6384e-05 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.541736+0000", 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003485391 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.541766+0000", 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0694000000000001e-05 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:6754 3.1a 3:5f18936f:::rbd_data.113535cec067.00000000000000db:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 333824~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.539293+0000", 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.99611534800000001, 2024-09-14T22:53:18.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048473309999999999, 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 6754 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539293+0000", 2024-09-14T22:53:18.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539293+0000", 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539294+0000", 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7300000000000005e-07 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539299+0000", 2024-09-14T22:53:18.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5549999999999996e-06 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539308+0000", 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2040000000000003e-06 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539313+0000", 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0130000000000001e-06 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539347+0000", 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4551999999999997e-05 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539418+0000", 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0784999999999994e-05 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 1", 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.539519+0000", 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010103300000000001 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.540459+0000", 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00094032100000000004 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.544049+0000", 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035891880000000001 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.544099+0000", 2024-09-14T22:53:18.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0243e-05 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.544141+0000", 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1664e-05 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:8804 3.14 3:2ff7c7f9:::rbd_data.113535cec067.00000000000000c6:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.892087+0000", 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.64332130099999996, 2024-09-14T22:53:18.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050241779999999998, 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 8804 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892087+0000", 2024-09-14T22:53:18.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892087+0000", 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892089+0000", 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7e-06 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892707+0000", 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061851300000000001 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892718+0000", 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.075e-05 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892723+0000", 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.062e-06 2024-09-14T22:53:18.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892776+0000", 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2862000000000003e-05 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892859+0000", 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3054999999999997e-05 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.893093+0000", 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023371399999999999 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.896207+0000", 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031141989999999998 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.897039+0000", 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083223300000000004 2024-09-14T22:53:18.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.897073+0000", 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3921000000000003e-05 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.897111+0000", 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8169e-05 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:8805 3.14 3:2ff7c7f9:::rbd_data.113535cec067.00000000000000c6:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.892736+0000", 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.64267285699999999, 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064966030000000001, 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 8805 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892736+0000", 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892736+0000", 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892736+0000", 2024-09-14T22:53:18.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892740+0000", 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1269999999999999e-06 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892746+0000", 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5470000000000002e-06 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.892751+0000", 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.13e-06 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.893998+0000", 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001247111 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.894079+0000", 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0446999999999997e-05 2024-09-14T22:53:18.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.894237+0000", 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015864300000000001 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.898842+0000", 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046048970000000002 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.899171+0000", 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000328568 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.899199+0000", 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.83e-05 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.899232+0000", 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3120000000000001e-05 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.622 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:8824 3.3 3:c415aa41:::rbd_data.113535cec067.00000000000000c3:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.895037+0000", 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.64037118199999998, 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045265990000000001, 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.623 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 8824 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895037+0000", 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895037+0000", 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895039+0000", 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2419999999999999e-06 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895578+0000", 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00053981799999999996 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895590+0000", 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1527e-05 2024-09-14T22:53:18.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895595+0000", 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1479999999999997e-06 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895644+0000", 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8999999999999998e-05 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895730+0000", 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5446999999999997e-05 2024-09-14T22:53:18.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895960+0000", 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023057500000000001 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.899385+0000", 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003425333 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.899454+0000", 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8722999999999999e-05 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.899518+0000", 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3385000000000004e-05 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.899564+0000", 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6400999999999998e-05 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.628 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:8825 3.3 3:c415aa41:::rbd_data.113535cec067.00000000000000c3:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.895610+0000", 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.63979829200000005, 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059902009999999997, 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 8825 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895610+0000", 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895610+0000", 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895611+0000", 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8000000000000005e-07 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895615+0000", 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1710000000000004e-06 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895622+0000", 2024-09-14T22:53:18.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.895629+0000", 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2550000000000003e-06 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.896827+0000", 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011980840000000001 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.896910+0000", 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999999e-05 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.897068+0000", 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000158968 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.899435+0000", 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023663709999999999 2024-09-14T22:53:18.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.901514+0000", 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020793729999999998 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.901560+0000", 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5670999999999997e-05 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.901600+0000", 2024-09-14T22:53:18.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0574000000000002e-05 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:8992 3.9 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.905433+0000", 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.62997581400000002, 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047254419999999998, 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 8992 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905433+0000", 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905433+0000", 2024-09-14T22:53:18.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905434+0000", 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.48e-06 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905926+0000", 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049136600000000002 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905936+0000", 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0195e-05 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905941+0000", 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1089999999999997e-06 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905996+0000", 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4608999999999998e-05 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.906076+0000", 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0912000000000001e-05 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 2", 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.906284+0000", 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020713799999999999 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.907972+0000", 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016887709999999999 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.910060+0000", 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020872519999999999 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.910115+0000", 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5087000000000003e-05 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.910158+0000", 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3523000000000003e-05 2024-09-14T22:53:18.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:8998 3.9 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.905954+0000", 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.62945431100000004, 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055862180000000004, 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:18.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 8998 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905954+0000", 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905954+0000", 2024-09-14T22:53:18.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905955+0000", 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905958+0000", 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6340000000000001e-06 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905964+0000", 2024-09-14T22:53:18.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1630000000000001e-06 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.905967+0000", 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0699999999999998e-06 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.907107+0000", 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011405600000000001 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.907193+0000", 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5416999999999999e-05 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 2", 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.907336+0000", 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014369200000000001 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.908123+0000", 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00078649899999999997 2024-09-14T22:53:18.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.911450+0000", 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033266979999999999 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.911503+0000", 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3158000000000002e-05 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.911540+0000", 2024-09-14T22:53:18.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7697999999999997e-05 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:12613 3.3 3:c4e5c6fc:::rbd_data.113535cec067.0000000000000155:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:18.365792+0000", 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.16961644300000001, 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042953649999999998, 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:18.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 12613 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.365792+0000", 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.365792+0000", 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:18.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.365793+0000", 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4799999999999997e-07 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.365798+0000", 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7049999999999998e-06 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.365809+0000", 2024-09-14T22:53:18.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1026000000000001e-05 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.365814+0000", 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3639999999999998e-06 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.365855+0000", 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1118999999999997e-05 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.365943+0000", 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8121999999999994e-05 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.370087+0000", 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041441810000000003 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:18.651 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:18.653 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:18.654 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:18.651+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:18.754 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:18.752+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:18.786 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:18.786 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:18.787 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T22:53:18.787 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:53:18.801 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:53:18.804 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:18.804 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:18.804 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:18.804 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:18.813 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:18.854 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:18.853+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:18.955 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:18.954+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:18.958 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:18.958 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:18.958 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:18.959 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:18.959 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:18.967 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:19.056 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:19.055+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:19.093 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:19.093 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:19.093 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 35, 2024-09-14T22:53:19.093 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:53:19.093 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:53:19.093 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:53:18.792696+0000", 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5801984, 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "noscrub", 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:53:19.094 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:53:19.095 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:19.096 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.097 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:19.098 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:19.099 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:19.100 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.101 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:19.102 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.103 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:19.104 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.105 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:19.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.9900000095367432, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.9900000095367432, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.75, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.75, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.107 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:19.108 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:19.109 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "34", 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 8, 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 34, 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:19.110 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:19.111 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:19.112 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.5, 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.5, 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.5, 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.5, 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.113 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:19.114 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:19.115 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:53:19.118 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:19.119 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:19.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:53:19.122 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:53:19.123 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:19.124 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 19, 2024-09-14T22:53:19.125 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.128 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:19.130 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:53:19.131 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:19.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.134 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.135 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:19.136 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:19.137 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:19.138 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.139 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:19.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:19.141 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.142 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.143 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.144 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:53:19.146 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:19.147 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 25, 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:19.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.149 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:53:19.152 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:19.153 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:19.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:19.158 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:19.159 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:53:19.160 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:19.161 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:19.162 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.163 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:19.164 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [], 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [], 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:53:19.165 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [], 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:53:19.166 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:53:19.167 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:53:19.167 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:53:19.167 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:19.167 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:19.170 INFO:tasks.thrashosds.thrasher:j is {'epoch': 35, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:53:18.792696+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 8, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.0', 'osds': [6, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:53:19.170 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:53:19.170 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:19.167+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4232.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:57.526704+0000", 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.587030165000002, 2024-09-14T22:53:19.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0092270879999999996, 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4232", 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/1486019522", 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.526704+0000", 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.526704+0000", 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.526710+0000", 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7300000000000002e-06 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.526718+0000", 2024-09-14T22:53:19.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0229999999999992e-06 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.526733+0000", 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5119999999999999e-05 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.526745+0000", 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2278e-05 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.526818+0000", 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2665999999999998e-05 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.526833+0000", 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5101e-05 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.530362+0000", 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035296839999999999 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.530500+0000", 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013763800000000001 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 2", 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.530639+0000", 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013937800000000001 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.531993+0000", 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013534899999999999 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.535856+0000", 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003863511 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.535893+0000", 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6328000000000002e-05 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:57.535931+0000", 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8141e-05 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:08.768982+0000", 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.344751784, 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053434659999999998, 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 10 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.768982+0000", 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.768982+0000", 2024-09-14T22:53:19.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.768983+0000", 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.215e-06 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.768988+0000", 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.544e-06 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.768995+0000", 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3799999999999996e-06 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.769002+0000", 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0029999999999997e-06 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.769052+0000", 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9638999999999997e-05 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.769058+0000", 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6160000000000001e-06 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.770632+0000", 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015742589999999999 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.770747+0000", 2024-09-14T22:53:19.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011556399999999999 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.770953+0000", 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020507500000000001 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.774240+0000", 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003287482 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.774267+0000", 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7396999999999999e-05 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.774299+0000", 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1127999999999997e-05 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:08.774326+0000", 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7163999999999999e-05 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1151 3.16 3:69c244c5:::rbd_data.1135de237b71.0000000000000064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.039837+0000", 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.0738974509999997, 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053973989999999998, 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1151 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.185 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039837+0000", 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039837+0000", 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039838+0000", 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8299999999999995e-07 2024-09-14T22:53:19.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039842+0000", 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8439999999999997e-06 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039855+0000", 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3103e-05 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039864+0000", 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5120000000000008e-06 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.039975+0000", 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111016 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040058+0000", 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3041999999999995e-05 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040163+0000", 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104547 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042120+0000", 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019571219999999999 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045153+0000", 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030329939999999998 2024-09-14T22:53:19.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045198+0000", 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5609999999999999e-05 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045234+0000", 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5626000000000001e-05 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1158 3.d e29/24 3:b02e2b95:::rbd_data.1135de237b71.000000000000008a:head v 29'10, mlcod=29'8)", 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040239+0000", 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.0734950169999999, 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004811208, 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040239+0000", 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040239+0000", 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040240+0000", 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040244+0000", 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6999999999999999e-06 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040255+0000", 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0654e-05 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040261+0000", 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1199999999999999e-06 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040309+0000", 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7559000000000003e-05 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040327+0000", 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8413000000000001e-05 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045012+0000", 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046850900000000003 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.045050+0000", 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8132e-05 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1153 3.1f e29/24 3:fe6d81c5:::rbd_data.1135de237b71.0000000000000066:head v 29'11, mlcod=29'9)", 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.040314+0000", 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.0734201089999997, 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046491350000000004, 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.195 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040314+0000", 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040314+0000", 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040314+0000", 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0200000000000002e-07 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040320+0000", 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4310000000000002e-06 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040328+0000", 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5189999999999992e-06 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040339+0000", 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0699e-05 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040386+0000", 2024-09-14T22:53:19.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7178999999999997e-05 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.040406+0000", 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9922999999999999e-05 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044923+0000", 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045166440000000002 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.044963+0000", 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0138000000000001e-05 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1161 3.2 e29/24 3:443dbe1d:::rbd_data.1135de237b71.0000000000000096:head v 29'7, mlcod=29'5)", 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.042524+0000", 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.0712099420000003, 2024-09-14T22:53:19.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005235768, 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042524+0000", 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042524+0000", 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042525+0000", 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6199999999999996e-07 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042530+0000", 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0950000000000003e-06 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042539+0000", 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.081e-06 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042544+0000", 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.583e-06 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042585+0000", 2024-09-14T22:53:19.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1041999999999998e-05 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.042604+0000", 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8958e-05 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.047727+0000", 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051231319999999999 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.047760+0000", 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3015000000000002e-05 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1187 3.16 3:6a245b79:::rbd_data.1135de237b71.0000000000000025:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:13.061682+0000", 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.0520521189999998, 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054896140000000003, 2024-09-14T22:53:19.205 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1187 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061682+0000", 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061682+0000", 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061683+0000", 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9400000000000004e-07 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061687+0000", 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3970000000000004e-06 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061697+0000", 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0044000000000001e-05 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061702+0000", 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9810000000000003e-06 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061746+0000", 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4056e-05 2024-09-14T22:53:19.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.061887+0000", 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000141237 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.062058+0000", 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017084399999999999 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.064501+0000", 2024-09-14T22:53:19.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002443003 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.067118+0000", 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0026165260000000001 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.067149+0000", 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1712999999999999e-05 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:13.067171+0000", 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2019e-05 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1654 3.1f e31/24 3:fa6b4654:::rbd_data.1135757ab9e2.000000000000000c:head v 31'13, mlcod=29'11)", 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:14.849417+0000", 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2643166299999997, 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076690500000000002, 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.849417+0000", 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.849417+0000", 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.849418+0000", 2024-09-14T22:53:19.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.61e-07 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.849424+0000", 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9739999999999999e-06 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.849436+0000", 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1737e-05 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.849443+0000", 2024-09-14T22:53:19.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9319999999999999e-06 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.849488+0000", 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5816999999999999e-05 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.849506+0000", 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7779999999999999e-05 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.856982+0000", 2024-09-14T22:53:19.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074758439999999997 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.857086+0000", 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104305 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:1664 3.2 e31/24 3:4158cdbe:::rbd_data.1135757ab9e2.000000000000000f:head v 31'11, mlcod=29'9)", 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:14.850730+0000", 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2630038130000001, 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063618040000000004, 2024-09-14T22:53:19.215 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.850730+0000", 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.850730+0000", 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.850730+0000", 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6e-07 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.850734+0000", 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0409999999999999e-06 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.850742+0000", 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4259999999999997e-06 2024-09-14T22:53:19.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.850746+0000", 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3610000000000003e-06 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.850790+0000", 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3284000000000001e-05 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.850809+0000", 2024-09-14T22:53:19.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9147000000000002e-05 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.857036+0000", 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062270609999999999 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.857092+0000", 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6224000000000002e-05 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:1671 3.c 3:34380f87:::rbd_data.1135757ab9e2.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:14.852501+0000", 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2612329080000002, 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049050320000000001, 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1671 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852501+0000", 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852501+0000", 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852502+0000", 2024-09-14T22:53:19.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5700000000000002e-07 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852505+0000", 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1520000000000001e-06 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852512+0000", 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852515+0000", 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5240000000000001e-06 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852553+0000", 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8077000000000002e-05 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852624+0000", 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1106999999999995e-05 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.852736+0000", 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011110700000000001 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.853574+0000", 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083887800000000004 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.857318+0000", 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003743349 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.857371+0000", 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2748999999999999e-05 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:14.857406+0000", 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5602999999999998e-05 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:8199 3.c 3:31ff52a7:::rbd_data.113535cec067.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 294912~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.750912+0000", 2024-09-14T22:53:19.225 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.3628221810000001, 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045223479999999998, 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 8199 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.750912+0000", 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.750912+0000", 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.750919+0000", 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7229999999999998e-06 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.751383+0000", 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00046439799999999998 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.751397+0000", 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3645999999999999e-05 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.751403+0000", 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9959999999999999e-06 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.751451+0000", 2024-09-14T22:53:19.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8313999999999999e-05 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.751541+0000", 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0288e-05 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.751687+0000", 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014613000000000001 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.753648+0000", 2024-09-14T22:53:19.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019609100000000002 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.755358+0000", 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001709976 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.755395+0000", 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7049999999999999e-05 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.755434+0000", 2024-09-14T22:53:19.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8917e-05 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:8800 3.1c 3:3d1e90c6:::rbd_data.113535cec067.00000000000000c0:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.886125+0000", 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.22760878, 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048899039999999996, 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.231 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 8800 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.886125+0000", 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.886125+0000", 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.886126+0000", 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4099999999999998e-07 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.886129+0000", 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.101e-06 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.886143+0000", 2024-09-14T22:53:19.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4287000000000001e-05 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.886148+0000", 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.899e-06 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.887157+0000", 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010083189999999999 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.887242+0000", 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5845000000000002e-05 2024-09-14T22:53:19.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.887408+0000", 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016512699999999999 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.890949+0000", 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035411919999999999 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.890962+0000", 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.365e-05 2024-09-14T22:53:19.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.890992+0000", 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0056000000000002e-05 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.891015+0000", 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2687000000000001e-05 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.236 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:9537 3.5 3:a4ea419a:::rbd_data.113535cec067.00000000000000cb:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.937441+0000", 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.176292755, 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048949919999999999, 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 9537 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.937441+0000", 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.937441+0000", 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.937442+0000", 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4e-07 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.937446+0000", 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5279999999999999e-06 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.937454+0000", 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9300000000000003e-06 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.937460+0000", 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8669999999999999e-06 2024-09-14T22:53:19.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.938557+0000", 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001097923 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.938640+0000", 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3033000000000003e-05 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.938817+0000", 2024-09-14T22:53:19.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017679100000000001 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.939995+0000", 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001177998 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.942272+0000", 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022765200000000002 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.942301+0000", 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9234000000000002e-05 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.942336+0000", 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5228000000000002e-05 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:9856 3.5 3:a0ac556c:::rbd_data.113535cec067.00000000000000e3:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.970771+0000", 2024-09-14T22:53:19.242 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.1429625750000001, 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052107630000000002, 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 9856 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.970771+0000", 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.970771+0000", 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.970773+0000", 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.269e-06 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.970779+0000", 2024-09-14T22:53:19.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7889999999999997e-06 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.970790+0000", 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0492999999999999e-05 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.970798+0000", 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6890000000000004e-06 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.970853+0000", 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5658000000000001e-05 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.970910+0000", 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6501e-05 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.971031+0000", 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000121669 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.973465+0000", 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002433789 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.975900+0000", 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024348769999999998 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.975946+0000", 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5779999999999999e-05 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.975982+0000", 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6248999999999998e-05 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:10688 3.5 3:a10e6f03:::rbd_data.113535cec067.00000000000000f0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:19.248 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:18.076987+0000", 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.036747305, 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045631819999999998, 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 10688 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.076987+0000", 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.076987+0000", 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.076988+0000", 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1540000000000001e-06 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077597+0000", 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00060893999999999998 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077625+0000", 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8388999999999999e-05 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077634+0000", 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7539999999999995e-06 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077690+0000", 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5952999999999998e-05 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077770+0000", 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9989999999999998e-05 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077983+0000", 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021304399999999999 2024-09-14T22:53:19.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.081400+0000", 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003416896 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.081448+0000", 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8155000000000002e-05 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.081509+0000", 2024-09-14T22:53:19.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0676e-05 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.081550+0000", 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1230999999999999e-05 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:10690 3.5 3:a3212938:::rbd_data.113535cec067.00000000000000f2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:18.077698+0000", 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.0360363239999999, 2024-09-14T22:53:19.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055300690000000003, 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 10690 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077698+0000", 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077698+0000", 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.077698+0000", 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078446+0000", 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00074758599999999995 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078458+0000", 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2489e-05 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078466+0000", 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9699999999999999e-06 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.079668+0000", 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0012022859999999999 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.079742+0000", 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3132000000000004e-05 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.079957+0000", 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021587200000000001 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.081612+0000", 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001654305 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.083171+0000", 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015590510000000001 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.083202+0000", 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1149000000000003e-05 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.083228+0000", 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5831999999999999e-05 2024-09-14T22:53:19.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:10691 3.5 3:a3212938:::rbd_data.113535cec067.00000000000000f2:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:18.078482+0000", 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.0352524519999999, 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075258310000000002, 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 10691 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078482+0000", 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078482+0000", 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078482+0000", 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1100000000000005e-07 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078486+0000", 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2499999999999998e-06 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078495+0000", 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000007e-06 2024-09-14T22:53:19.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.078501+0000", 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.080615+0000", 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002113579 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.080659+0000", 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3958000000000002e-05 2024-09-14T22:53:19.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.080826+0000", 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016716300000000001 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.081682+0000", 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085626300000000001 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.085912+0000", 2024-09-14T22:53:19.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042294239999999999 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.085967+0000", 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5747e-05 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.086007+0000", 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9901000000000003e-05 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.265 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:10839 3.c 3:33740b19:::rbd_data.113535cec067.00000000000000f8:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:18.111275+0000", 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.00245899, 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00485557, 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 10839 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.266 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111275+0000", 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111275+0000", 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111277+0000", 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5340000000000001e-06 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111771+0000", 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049434299999999995 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111786+0000", 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5031000000000001e-05 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111793+0000", 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6760000000000001e-06 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111850+0000", 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7073000000000001e-05 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111947+0000", 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7687000000000002e-05 2024-09-14T22:53:19.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.112167+0000", 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021991099999999999 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.113841+0000", 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016737810000000001 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.116054+0000", 2024-09-14T22:53:19.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022126279999999999 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.116102+0000", 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8318000000000001e-05 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.116131+0000", 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8588000000000001e-05 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:10840 3.c 3:33740b19:::rbd_data.113535cec067.00000000000000f8:head [delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:18.111812+0000", 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.001921922, 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047658850000000001, 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 10840 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111812+0000", 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111812+0000", 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111813+0000", 2024-09-14T22:53:19.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8400000000000004e-07 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111815+0000", 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7750000000000001e-06 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111820+0000", 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9729999999999997e-06 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.111824+0000", 2024-09-14T22:53:19.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5090000000000002e-06 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.113027+0000", 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001202844 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.113074+0000", 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7720999999999999e-05 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.113200+0000", 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012556000000000001 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.114161+0000", 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00096142899999999997 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.116511+0000", 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023499390000000001 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.116542+0000", 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.046e-05 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.116578+0000", 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090999999999997e-05 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "pg_update_log_missing(3.1f epoch 35/24 rep_tid 603 entries 35'66 (0'0) error 3:fa79ae32:::rbd_data.113535cec067.00000000000001de:head by client.4405.0:16485 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 34'64)", 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:18.882725+0000", 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.231008512, 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046044060000000001, 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.882725+0000", 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.882725+0000", 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.882726+0000", 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1800000000000002e-07 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.882732+0000", 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.229e-06 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.882750+0000", 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7822999999999999e-05 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.882772+0000", 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2209e-05 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.882846+0000", 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3535000000000007e-05 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:18.887330+0000", 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004484292 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:19.283 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:19.284 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:19.284 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:19.284 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:19.286 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T22:53:19.286 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:19.285+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:19.387 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:19.385+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:19.388 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:19.389 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:19.389 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:19.389 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:19.397 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:19.418 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:19.419 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:19.436 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:19.436 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:19.454 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:19.454 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:19.471 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:19.471 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:19.488 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:19.486+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:19.488 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:19.488 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:19.500 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:19.500 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:19.505 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:19.505 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:19.508 INFO:teuthology.orchestra.run.smithi076.stdout:621 write 0x3336667 thru 0x333c151 (0x5aeb bytes) 2024-09-14T22:53:19.509 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:19.531 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:19.532 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:19.577 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:19.577 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:19.593 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:19.592+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:19.642 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:19.643 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:53:19.654 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-09-14T22:53:18.798445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19414160,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3714,"num_read_kb":3457,"num_write":996,"num_write_kb":6845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229112,"kb_used_data":4500,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745608,"statfs":{"total":773094113280,"available":772859502592,"internally_reserved":0,"allocated":4608000,"data_stored":2843982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18823708,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":3564,"num_read_kb":3331,"num_write":829,"num_write_kb":4421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.012093"},"pg_stats":[{"pgid":"3.1f","version":"29'11","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.774101+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:53:14.774101+0000","last_peered":"2024-09-14T22:53:14.774101+0000","last_clean":"2024-09-14T22:53:14.774101+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:53:14.774101+0000","last_undegraded":"2024-09-14T22:53:14.774101+0000","last_fullsized":"2024-09-14T22:53:14.774101+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:24.637386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":2,"num_write":11,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'3","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780213+0000","last_change":"2024-09-14T22:53:08.761968+0000","last_active":"2024-09-14T22:53:14.780213+0000","last_peered":"2024-09-14T22:53:14.780213+0000","last_clean":"2024-09-14T22:53:14.780213+0000","last_became_active":"2024-09-14T22:53:06.755533+0000","last_became_peered":"2024-09-14T22:53:06.755533+0000","last_unstale":"2024-09-14T22:53:14.780213+0000","last_undegraded":"2024-09-14T22:53:14.780213+0000","last_fullsized":"2024-09-14T22:53:14.780213+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:04:57.466028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098261599999999996,"stat_sum":{"num_bytes":182168,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780679+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:53:14.780679+0000","last_peered":"2024-09-14T22:53:14.780679+0000","last_clean":"2024-09-14T22:53:14.780679+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:53:14.780679+0000","last_undegraded":"2024-09-14T22:53:14.780679+0000","last_fullsized":"2024-09-14T22:53:14.780679+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:14:51.171877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"30'14","reported_seq":63,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.789591+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:53:14.789591+0000","last_peered":"2024-09-14T22:53:14.789591+0000","last_clean":"2024-09-14T22:53:14.789591+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:53:14.789591+0000","last_undegraded":"2024-09-14T22:53:14.789591+0000","last_fullsized":"2024-09-14T22:53:14.789591+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:40:38.748416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":1500374,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":44,"num_write":13,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"29'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.773919+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:53:14.773919+0000","last_peered":"2024-09-14T22:53:14.773919+0000","last_clean":"2024-09-14T22:53:14.773919+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:53:14.773919+0000","last_undegraded":"2024-09-14T22:53:14.773919+0000","last_fullsized":"2024-09-14T22:53:14.773919+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:04.637641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":1835008,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":50,"num_write":11,"num_write_kb":212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"29'6","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.799168+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:53:14.799168+0000","last_peered":"2024-09-14T22:53:14.799168+0000","last_clean":"2024-09-14T22:53:14.799168+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:53:14.799168+0000","last_undegraded":"2024-09-14T22:53:14.799168+0000","last_fullsized":"2024-09-14T22:53:14.799168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:38:10.467141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":105,"num_read_kb":104,"num_write":6,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"31'16","reported_seq":102,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.799480+0000","last_change":"2024-09-14T22:53:08.766652+0000","last_active":"2024-09-14T22:53:14.799480+0000","last_peered":"2024-09-14T22:53:14.799480+0000","last_clean":"2024-09-14T22:53:14.799480+0000","last_became_active":"2024-09-14T22:53:06.760167+0000","last_became_peered":"2024-09-14T22:53:06.760167+0000","last_unstale":"2024-09-14T22:53:14.799480+0000","last_undegraded":"2024-09-14T22:53:14.799480+0000","last_fullsized":"2024-09-14T22:53:14.799480+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:31:51.413579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":71300,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":164,"num_read_kb":137,"num_write":15,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"29'12","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780216+0000","last_change":"2024-09-14T22:53:08.761429+0000","last_active":"2024-09-14T22:53:14.780216+0000","last_peered":"2024-09-14T22:53:14.780216+0000","last_clean":"2024-09-14T22:53:14.780216+0000","last_became_active":"2024-09-14T22:53:06.759353+0000","last_became_peered":"2024-09-14T22:53:06.759353+0000","last_unstale":"2024-09-14T22:53:14.780216+0000","last_undegraded":"2024-09-14T22:53:14.780216+0000","last_fullsized":"2024-09-14T22:53:14.780216+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:18:23.425284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062866099999999998,"stat_sum":{"num_bytes":786432,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":12,"num_write_kb":178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"30'28","reported_seq":187,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.806924+0000","last_change":"2024-09-14T22:53:08.777352+0000","last_active":"2024-09-14T22:53:14.806924+0000","last_peered":"2024-09-14T22:53:14.806924+0000","last_clean":"2024-09-14T22:53:14.806924+0000","last_became_active":"2024-09-14T22:53:06.758502+0000","last_became_peered":"2024-09-14T22:53:06.758502+0000","last_unstale":"2024-09-14T22:53:14.806924+0000","last_undegraded":"2024-09-14T22:53:14.806924+0000","last_fullsized":"2024-09-14T22:53:14.806924+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:44:11.819419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":52,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":324,"num_write":28,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"29'5","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780841+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:53:14.780841+0000","last_peered":"2024-09-14T22:53:14.780841+0000","last_clean":"2024-09-14T22:53:14.780841+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:53:14.780841+0000","last_undegraded":"2024-09-14T22:53:14.780841+0000","last_fullsized":"2024-09-14T22:53:14.780841+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:56:07.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":786432,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"29'2","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.783220+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:53:14.783220+0000","last_peered":"2024-09-14T22:53:14.783220+0000","last_clean":"2024-09-14T22:53:14.783220+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:53:14.783220+0000","last_undegraded":"2024-09-14T22:53:14.783220+0000","last_fullsized":"2024-09-14T22:53:14.783220+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:29:25.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":993212,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"31'25","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.782535+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:53:14.782535+0000","last_peered":"2024-09-14T22:53:14.782535+0000","last_clean":"2024-09-14T22:53:14.782535+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:53:14.782535+0000","last_undegraded":"2024-09-14T22:53:14.782535+0000","last_fullsized":"2024-09-14T22:53:14.782535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:29:06.809563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":1823276,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":66,"num_write":25,"num_write_kb":401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-14T22:53:13.812317+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:53:13.812317+0000","last_peered":"2024-09-14T22:53:13.812317+0000","last_clean":"2024-09-14T22:53:13.812317+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:53:13.812317+0000","last_undegraded":"2024-09-14T22:53:13.812317+0000","last_fullsized":"2024-09-14T22:53:13.812317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"29'197","reported_seq":229,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.773675+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:53:14.773675+0000","last_peered":"2024-09-14T22:53:14.773675+0000","last_clean":"2024-09-14T22:53:14.773675+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:53:14.773675+0000","last_undegraded":"2024-09-14T22:53:14.773675+0000","last_fullsized":"2024-09-14T22:53:14.773675+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:59:54.854145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":47,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":948,"num_read_kb":754,"num_write":319,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.776163+0000","last_change":"2024-09-14T22:52:59.538561+0000","last_active":"2024-09-14T22:53:14.776163+0000","last_peered":"2024-09-14T22:53:14.776163+0000","last_clean":"2024-09-14T22:53:14.776163+0000","last_became_active":"2024-09-14T22:52:57.530436+0000","last_became_peered":"2024-09-14T22:52:57.530436+0000","last_unstale":"2024-09-14T22:53:14.776163+0000","last_undegraded":"2024-09-14T22:53:14.776163+0000","last_fullsized":"2024-09-14T22:53:14.776163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:30:29.562282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"28'14","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780248+0000","last_change":"2024-09-14T22:53:08.762000+0000","last_active":"2024-09-14T22:53:14.780248+0000","last_peered":"2024-09-14T22:53:14.780248+0000","last_clean":"2024-09-14T22:53:14.780248+0000","last_became_active":"2024-09-14T22:53:06.761953+0000","last_became_peered":"2024-09-14T22:53:06.761953+0000","last_unstale":"2024-09-14T22:53:14.780248+0000","last_undegraded":"2024-09-14T22:53:14.780248+0000","last_fullsized":"2024-09-14T22:53:14.780248+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:06:52.452675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":14,"num_write_kb":225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.783409+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:14.783409+0000","last_peered":"2024-09-14T22:53:14.783409+0000","last_clean":"2024-09-14T22:53:14.783409+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:14.783409+0000","last_undegraded":"2024-09-14T22:53:14.783409+0000","last_fullsized":"2024-09-14T22:53:14.783409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"29'7","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.776069+0000","last_change":"2024-09-14T22:53:08.766672+0000","last_active":"2024-09-14T22:53:14.776069+0000","last_peered":"2024-09-14T22:53:14.776069+0000","last_clean":"2024-09-14T22:53:14.776069+0000","last_became_active":"2024-09-14T22:53:06.755437+0000","last_became_peered":"2024-09-14T22:53:06.755437+0000","last_unstale":"2024-09-14T22:53:14.776069+0000","last_undegraded":"2024-09-14T22:53:14.776069+0000","last_fullsized":"2024-09-14T22:53:14.776069+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:02:59.678153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":2121636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":27,"num_write":7,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.776079+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:53:14.776079+0000","last_peered":"2024-09-14T22:53:14.776079+0000","last_clean":"2024-09-14T22:53:14.776079+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:53:14.776079+0000","last_undegraded":"2024-09-14T22:53:14.776079+0000","last_fullsized":"2024-09-14T22:53:14.776079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"29'2","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780829+0000","last_change":"2024-09-14T22:53:08.771398+0000","last_active":"2024-09-14T22:53:14.780829+0000","last_peered":"2024-09-14T22:53:14.780829+0000","last_clean":"2024-09-14T22:53:14.780829+0000","last_became_active":"2024-09-14T22:53:06.758572+0000","last_became_peered":"2024-09-14T22:53:06.758572+0000","last_unstale":"2024-09-14T22:53:14.780829+0000","last_undegraded":"2024-09-14T22:53:14.780829+0000","last_fullsized":"2024-09-14T22:53:14.780829+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:24:17.407592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":1572864,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"30'15","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.773475+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:53:14.773475+0000","last_peered":"2024-09-14T22:53:14.773475+0000","last_clean":"2024-09-14T22:53:14.773475+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:53:14.773475+0000","last_undegraded":"2024-09-14T22:53:14.773475+0000","last_fullsized":"2024-09-14T22:53:14.773475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:09:28.719445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":262144,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":54,"num_write":15,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780573+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:53:14.780573+0000","last_peered":"2024-09-14T22:53:14.780573+0000","last_clean":"2024-09-14T22:53:14.780573+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:53:14.780573+0000","last_undegraded":"2024-09-14T22:53:14.780573+0000","last_fullsized":"2024-09-14T22:53:14.780573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"30'13","reported_seq":41,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-14T22:53:13.881061+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:53:13.881061+0000","last_peered":"2024-09-14T22:53:13.881061+0000","last_clean":"2024-09-14T22:53:13.881061+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:53:13.881061+0000","last_undegraded":"2024-09-14T22:53:13.881061+0000","last_fullsized":"2024-09-14T22:53:13.881061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:08:41.807854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":842795,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":12,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.773142+0000","last_change":"2024-09-14T22:53:00.039871+0000","last_active":"2024-09-14T22:53:14.773142+0000","last_peered":"2024-09-14T22:53:14.773142+0000","last_clean":"2024-09-14T22:53:14.773142+0000","last_became_active":"2024-09-14T22:52:57.694658+0000","last_became_peered":"2024-09-14T22:52:57.694658+0000","last_unstale":"2024-09-14T22:53:14.773142+0000","last_undegraded":"2024-09-14T22:53:14.773142+0000","last_fullsized":"2024-09-14T22:53:14.773142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:41:09.229943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"29'2","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.773431+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:53:14.773431+0000","last_peered":"2024-09-14T22:53:14.773431+0000","last_clean":"2024-09-14T22:53:14.773431+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:53:14.773431+0000","last_undegraded":"2024-09-14T22:53:14.773431+0000","last_fullsized":"2024-09-14T22:53:14.773431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:20:25.436348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.782798+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:14.782798+0000","last_peered":"2024-09-14T22:53:14.782798+0000","last_clean":"2024-09-14T22:53:14.782798+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:14.782798+0000","last_undegraded":"2024-09-14T22:53:14.782798+0000","last_fullsized":"2024-09-14T22:53:14.782798+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"29'9","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780363+0000","last_change":"2024-09-14T22:53:08.761713+0000","last_active":"2024-09-14T22:53:14.780363+0000","last_peered":"2024-09-14T22:53:14.780363+0000","last_clean":"2024-09-14T22:53:14.780363+0000","last_became_active":"2024-09-14T22:53:06.763198+0000","last_became_peered":"2024-09-14T22:53:06.763198+0000","last_unstale":"2024-09-14T22:53:14.780363+0000","last_undegraded":"2024-09-14T22:53:14.780363+0000","last_fullsized":"2024-09-14T22:53:14.780363+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:24:24.832661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":9,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780580+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:53:14.780580+0000","last_peered":"2024-09-14T22:53:14.780580+0000","last_clean":"2024-09-14T22:53:14.780580+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:53:14.780580+0000","last_undegraded":"2024-09-14T22:53:14.780580+0000","last_fullsized":"2024-09-14T22:53:14.780580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.782879+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:14.782879+0000","last_peered":"2024-09-14T22:53:14.782879+0000","last_clean":"2024-09-14T22:53:14.782879+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:14.782879+0000","last_undegraded":"2024-09-14T22:53:14.782879+0000","last_fullsized":"2024-09-14T22:53:14.782879+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"30'91","reported_seq":623,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.812114+0000","last_change":"2024-09-14T22:53:08.761436+0000","last_active":"2024-09-14T22:53:14.812114+0000","last_peered":"2024-09-14T22:53:14.812114+0000","last_clean":"2024-09-14T22:53:14.812114+0000","last_became_active":"2024-09-14T22:53:06.757033+0000","last_became_peered":"2024-09-14T22:53:06.757033+0000","last_unstale":"2024-09-14T22:53:14.812114+0000","last_undegraded":"2024-09-14T22:53:14.812114+0000","last_fullsized":"2024-09-14T22:53:14.812114+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:51.751076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1477,"num_read_kb":1297,"num_write":85,"num_write_kb":193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.773298+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:53:14.773298+0000","last_peered":"2024-09-14T22:53:14.773298+0000","last_clean":"2024-09-14T22:53:14.773298+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:53:14.773298+0000","last_undegraded":"2024-09-14T22:53:14.773298+0000","last_fullsized":"2024-09-14T22:53:14.773298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:57:13.596447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"30'4","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-14T22:53:13.812386+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:53:13.812386+0000","last_peered":"2024-09-14T22:53:13.812386+0000","last_clean":"2024-09-14T22:53:13.812386+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:53:13.812386+0000","last_undegraded":"2024-09-14T22:53:13.812386+0000","last_fullsized":"2024-09-14T22:53:13.812386+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:57:28.856661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"28'4","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780176+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:53:14.780176+0000","last_peered":"2024-09-14T22:53:14.780176+0000","last_clean":"2024-09-14T22:53:14.780176+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:53:14.780176+0000","last_undegraded":"2024-09-14T22:53:14.780176+0000","last_fullsized":"2024-09-14T22:53:14.780176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:31:36.835111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"31'14","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.780969+0000","last_change":"2024-09-14T22:53:08.770726+0000","last_active":"2024-09-14T22:53:14.780969+0000","last_peered":"2024-09-14T22:53:14.780969+0000","last_clean":"2024-09-14T22:53:14.780969+0000","last_became_active":"2024-09-14T22:53:06.761991+0000","last_became_peered":"2024-09-14T22:53:06.761991+0000","last_unstale":"2024-09-14T22:53:14.780969+0000","last_undegraded":"2024-09-14T22:53:14.780969+0000","last_fullsized":"2024-09-14T22:53:14.780969+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:28:33.791837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":131351,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":43,"num_write":19,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"29'14","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.783359+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:53:14.783359+0000","last_peered":"2024-09-14T22:53:14.783359+0000","last_clean":"2024-09-14T22:53:14.783359+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:53:14.783359+0000","last_undegraded":"2024-09-14T22:53:14.783359+0000","last_fullsized":"2024-09-14T22:53:14.783359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:27:30.458170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":3130654,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":64,"num_write":14,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"30'16","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.783310+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:53:14.783310+0000","last_peered":"2024-09-14T22:53:14.783310+0000","last_clean":"2024-09-14T22:53:14.783310+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:53:14.783310+0000","last_undegraded":"2024-09-14T22:53:14.783310+0000","last_fullsized":"2024-09-14T22:53:14.783310+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:08.485945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":222770,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":71,"num_write":16,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"29'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.783038+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:53:14.783038+0000","last_peered":"2024-09-14T22:53:14.783038+0000","last_clean":"2024-09-14T22:53:14.783038+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:53:14.783038+0000","last_undegraded":"2024-09-14T22:53:14.783038+0000","last_fullsized":"2024-09-14T22:53:14.783038+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:06:09.934975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"29'4","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-14T22:53:13.812351+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:53:13.812351+0000","last_peered":"2024-09-14T22:53:13.812351+0000","last_clean":"2024-09-14T22:53:13.812351+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:53:13.812351+0000","last_undegraded":"2024-09-14T22:53:13.812351+0000","last_fullsized":"2024-09-14T22:53:13.812351+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:21:34.093329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"29'7","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.782875+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:53:14.782875+0000","last_peered":"2024-09-14T22:53:14.782875+0000","last_clean":"2024-09-14T22:53:14.782875+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:53:14.782875+0000","last_undegraded":"2024-09-14T22:53:14.782875+0000","last_fullsized":"2024-09-14T22:53:14.782875+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:25.946322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":786432,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":7,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"30'84","reported_seq":116,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.773511+0000","last_change":"2024-09-14T22:53:08.776638+0000","last_active":"2024-09-14T22:53:14.773511+0000","last_peered":"2024-09-14T22:53:14.773511+0000","last_clean":"2024-09-14T22:53:14.773511+0000","last_became_active":"2024-09-14T22:53:06.760891+0000","last_became_peered":"2024-09-14T22:53:06.760891+0000","last_unstale":"2024-09-14T22:53:14.773511+0000","last_undegraded":"2024-09-14T22:53:14.773511+0000","last_fullsized":"2024-09-14T22:53:14.773511+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:13:07.124133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":917556,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":358,"num_read_kb":291,"num_write":138,"num_write_kb":410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"29'25","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T22:53:14.782978+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:53:14.782978+0000","last_peered":"2024-09-14T22:53:14.782978+0000","last_clean":"2024-09-14T22:53:14.782978+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:53:14.782978+0000","last_undegraded":"2024-09-14T22:53:14.782978+0000","last_fullsized":"2024-09-14T22:53:14.782978+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:35.456078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":857219,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":23,"num_write":25,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":18823773,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":3598,"num_read_kb":3357,"num_write":835,"num_write_kb":4427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966656,"data_stored":770964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":664,"ondisk_log_size":664,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1228800,"data_stored":978456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":327680,"data_stored":125118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":309797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":417792,"data_stored":206093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":135442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":344064,"data_stored":141882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542154,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":364544,"data_stored":147731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":799463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":83993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":22610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":16780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":55069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":141721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":184676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":315392,"data_stored":263218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:53:19.656 INFO:tasks.thrashosds.thrasher:Setting 2.6 to [5, 2] 2024-09-14T22:53:19.656 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.6', '5', '2'] 2024-09-14T22:53:19.656 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.6 5 2 2024-09-14T22:53:19.659 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:19.659 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:19.668 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:19.699 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:19.698+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:19.724 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:19.725 INFO:teuthology.orchestra.run.smithi076.stdout:623 punch from 0x58f7fff to 0x58ffcc9, (0x7cca bytes) 2024-09-14T22:53:19.725 INFO:teuthology.orchestra.run.smithi076.stdout:624 writesame 0xcd2636d thru 0xcd355e7 data_size 0x41(0xf27b bytes) 2024-09-14T22:53:19.777 INFO:teuthology.orchestra.run.smithi076.stdout:629 read 0x3ffc27a thru 0x4009e2d (0xdbb4 bytes) 2024-09-14T22:53:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:631 punch from 0x858ab77 to 0x859027e, (0x5707 bytes) 2024-09-14T22:53:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:632 read 0x5b07386 thru 0x5b0b0d8 (0x3d53 bytes) 2024-09-14T22:53:19.781 INFO:teuthology.orchestra.run.smithi076.stdout:633 read 0x12de88c thru 0x12ec587 (0xdcfc bytes) 2024-09-14T22:53:19.782 INFO:teuthology.orchestra.run.smithi076.stdout:634 clone 7 order 22 su 65536 sc 8 2024-09-14T22:53:19.790 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:19.789+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:19.817 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:19.817 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:19.826 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:19.892 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:19.889+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:19.929 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:19.929 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:19.937 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:19.991 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:19.990+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:20.074 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone6 from 0xeb79b5c (overlap 0xeb79b5c) to 0x5e30af1 2024-09-14T22:53:20.090 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:20.090 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:20.092 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:20.091+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:20.099 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:20.193 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:20.191+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:20.233 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:20.233 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:20.244 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:20.293 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:20.292+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:20.378 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:20.378 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:20.389 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:20.430 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:20.429+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:20.495 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:20.493+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:20.574 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:20.574 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:20.584 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:20.596 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:20.594+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:20.692 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:20.692 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:20.696 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:20.695+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:20.700 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:20.797 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:20.796+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:20.804 INFO:teuthology.orchestra.run.smithi076.stderr:set 2.6 pg_upmap mapping to [5,2] 2024-09-14T22:53:20.818 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:53:20.867 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:20.868 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:20.882 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:20.898 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:20.897+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:20.999 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:20.998+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:21.093 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:21.093 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:21.100 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:21.099+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:21.102 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:21.202 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:21.200+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:21.283 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:21.284 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:21.297 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:21.303 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:21.302+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:21.404 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:21.402+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:21.438 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:21.447 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:21.531 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:21.530+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:21.593 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:21.594 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:21.594 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:21.604 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:21.645 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:21.644+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:21.707 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:21.705+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:21.736 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:21.736 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:21.754 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:21.754 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:21.770 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:21.770 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:21.771 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:21.781 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:21.789 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:21.789 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:21.805 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:21.805 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:21.807 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:21.806+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:21.817 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:53:21.827 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:21.827 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:21.847 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:21.848 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:21.857 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T22:53:21.874 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:21.919 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:21.920 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:21.921 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:21.920+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:21.933 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:22.009 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:22.008+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:22.110 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:22.109+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30305 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1349632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215671793, 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051104002000000003, 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.123 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30305 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8799999999999998e-07 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904388+0000", 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7000000000000008e-06 2024-09-14T22:53:22.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904395+0000", 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904401+0000", 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.147e-06 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955309+0000", 2024-09-14T22:53:22.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.050908627999999997 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955388+0000", 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8615999999999994e-05 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955484+0000", 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6323000000000006e-05 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.127 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21564428299999999, 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.128 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:53:22.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:53:22.132 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215623747, 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.133 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:53:22.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215593748, 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.138 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:22.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:53:22.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.142 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215562595, 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.143 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:22.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.147 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21553993900000001, 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.148 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:22.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21551943800000001, 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:53:22.152 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:53:22.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215499677, 2024-09-14T22:53:22.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:53:22.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21547889200000001, 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.162 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:22.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:22.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:53:22.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21544648, 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:53:22.167 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21542508099999999, 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:53:22.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215363682, 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:53:22.176 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:53:22.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215323769, 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.181 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:53:22.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21529401300000001, 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:53:22.185 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:53:22.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.189 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21526377799999999, 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:53:22.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.194 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21523903899999999, 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:53:22.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:53:22.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:53:22.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.199 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215215609, 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:53:22.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21511987299999999, 2024-09-14T22:53:22.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:22.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:53:22.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.215090951, 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:53:22.209 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:53:22.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.21506762600000001, 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:22.214 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:53:22.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:22.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.218 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.219 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.219 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:22.219 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:22.219 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:22.220 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:22.221 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:22.219+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:22.321 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:22.320+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:22.357 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:22.357 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:22.357 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:22.357 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:22.366 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:22.422 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:22.421+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:22.509 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:22.510 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:22.510 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:22.510 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:22.510 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:22.522 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:22.523 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:22.522+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:22.625 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:22.623+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40556 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1980416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.330322+0000", 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33905410200000002, 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091170149999999991, 2024-09-14T22:53:22.672 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40556 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330322+0000", 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330322+0000", 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330323+0000", 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330329+0000", 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0419999999999999e-06 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330337+0000", 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0520000000000002e-06 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330343+0000", 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339328+0000", 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089851489999999996 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339390+0000", 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1438000000000004e-05 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339440+0000", 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9736e-05 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.676 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40557 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 2004992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.330353+0000", 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33902404000000003, 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0092882799999999995, 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40557 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330353+0000", 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330353+0000", 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330353+0000", 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7800000000000001e-07 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330358+0000", 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.69e-06 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330364+0000", 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6470000000000001e-06 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330370+0000", 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339483+0000", 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091130910000000002 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339565+0000", 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2556e-05 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339641+0000", 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5597000000000001e-05 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.680 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40558 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.330377+0000", 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33900008799999998, 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0094644729999999993, 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40558 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330377+0000", 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330377+0000", 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330377+0000", 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5699999999999998e-07 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330379+0000", 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.181e-06 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330390+0000", 2024-09-14T22:53:22.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1100999999999999e-05 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330396+0000", 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4140000000000002e-06 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339682+0000", 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0092858560000000003 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339770+0000", 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8343000000000004e-05 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339841+0000", 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1120999999999999e-05 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40559 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 2054144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.330404+0000", 2024-09-14T22:53:22.685 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33897301299999999, 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096164890000000006, 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40559 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330404+0000", 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330404+0000", 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330404+0000", 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.44e-07 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330406+0000", 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2589999999999999e-06 2024-09-14T22:53:22.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330413+0000", 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2439999999999998e-06 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330418+0000", 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0560000000000004e-06 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339872+0000", 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0094545900000000006 2024-09-14T22:53:22.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.339951+0000", 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9013999999999999e-05 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.340020+0000", 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8782000000000007e-05 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40560 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 2078720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.330424+0000", 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33895228100000002, 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0097632199999999995, 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40560 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330424+0000", 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330424+0000", 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330425+0000", 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2100000000000002e-07 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330427+0000", 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9740000000000001e-06 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330433+0000", 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.942e-06 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.330437+0000", 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6489999999999999e-06 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.340049+0000", 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096116799999999992 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.340118+0000", 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9060999999999995e-05 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.340188+0000", 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9492999999999993e-05 2024-09-14T22:53:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40609 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1439744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.332193+0000", 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33718376700000002, 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091349889999999996, 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.694 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40609 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332193+0000", 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332193+0000", 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332194+0000", 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6000000000000003e-07 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332197+0000", 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4209999999999999e-06 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332204+0000", 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7139999999999996e-06 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332209+0000", 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6860000000000003e-06 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341193+0000", 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089834540000000001 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341262+0000", 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8973000000000005e-05 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341328+0000", 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5981000000000004e-05 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40642 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1464320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.332224+0000", 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33715293200000002, 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0093184029999999994, 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.698 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40642 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332224+0000", 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332224+0000", 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332224+0000", 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6499999999999999e-07 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332233+0000", 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332372+0000", 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000139202 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332380+0000", 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3490000000000003e-06 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341374+0000", 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089944829999999993 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341457+0000", 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2922000000000003e-05 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341542+0000", 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5031000000000006e-05 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40643 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1488896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.332392+0000", 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33698498900000001, 2024-09-14T22:53:22.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0093336070000000007, 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40643 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332392+0000", 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332392+0000", 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332392+0000", 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332396+0000", 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.552e-06 2024-09-14T22:53:22.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332405+0000", 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0350000000000007e-06 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332411+0000", 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0249999999999999e-06 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341568+0000", 2024-09-14T22:53:22.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091574919999999997 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341650+0000", 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1475000000000002e-05 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341725+0000", 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5352000000000006e-05 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.706 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40644 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1513472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.332418+0000", 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33695816299999998, 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0094904020000000002, 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40644 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332418+0000", 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332418+0000", 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332419+0000", 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8800000000000003e-07 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332421+0000", 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3149999999999999e-06 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332428+0000", 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6969999999999996e-06 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332433+0000", 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2089999999999996e-06 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341772+0000", 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0093390450000000007 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341844+0000", 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2055e-05 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341909+0000", 2024-09-14T22:53:22.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4592999999999996e-05 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40645 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1538048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.332440+0000", 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33693630200000002, 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096683180000000004, 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40645 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332440+0000", 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332440+0000", 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332441+0000", 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9099999999999999e-07 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332443+0000", 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2079999999999999e-06 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332449+0000", 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3210000000000001e-06 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332454+0000", 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9779999999999996e-06 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341960+0000", 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0095058009999999995 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342035+0000", 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4980000000000004e-05 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342109+0000", 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3639000000000004e-05 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40646 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1562624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.332462+0000", 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33691415600000002, 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.009855908, 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.715 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40646 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332462+0000", 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332462+0000", 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332463+0000", 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.34e-07 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332465+0000", 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1770000000000001e-06 2024-09-14T22:53:22.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332473+0000", 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.332478+0000", 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9819999999999999e-06 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342154+0000", 2024-09-14T22:53:22.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096755780000000007 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342235+0000", 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1520999999999994e-05 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342318+0000", 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3305000000000003e-05 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40680 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1636352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.333547+0000", 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33582928400000001, 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091293529999999998, 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40680 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.720 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333547+0000", 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333547+0000", 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333548+0000", 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6199999999999998e-07 2024-09-14T22:53:22.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333551+0000", 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6429999999999999e-06 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333558+0000", 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1250000000000004e-06 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333563+0000", 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6189999999999999e-06 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342541+0000", 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089778879999999998 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342610+0000", 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342677+0000", 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6758e-05 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40681 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1660928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.333571+0000", 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33580549599999998, 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0092665349999999994, 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40681 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333571+0000", 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333571+0000", 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333571+0000", 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7300000000000002e-07 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333574+0000", 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2479999999999999e-06 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333580+0000", 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1210000000000003e-06 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333585+0000", 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342705+0000", 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091203729999999993 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342771+0000", 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6444000000000003e-05 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342838+0000", 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6146e-05 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40682 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1611776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.333595+0000", 2024-09-14T22:53:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33578171800000001, 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0094186960000000007, 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40682 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333595+0000", 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333595+0000", 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333595+0000", 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9399999999999995e-07 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333598+0000", 2024-09-14T22:53:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6199999999999999e-06 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333605+0000", 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3679999999999996e-06 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333610+0000", 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3359999999999997e-06 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342865+0000", 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0092552469999999994 2024-09-14T22:53:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.342941+0000", 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5860999999999998e-05 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.343014+0000", 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2769999999999996e-05 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40689 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1685504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.333616+0000", 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33576021900000003, 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0095712090000000007, 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40689 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333616+0000", 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333616+0000", 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333617+0000", 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1399999999999997e-07 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333619+0000", 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.725e-06 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333623+0000", 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5970000000000002e-06 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.333627+0000", 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3630000000000004e-06 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.343043+0000", 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0094159770000000007 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.343113+0000", 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9688000000000001e-05 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.343188+0000", 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4444999999999996e-05 2024-09-14T22:53:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40719 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1808384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.334788+0000", 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33458881299999998, 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090936379999999994, 2024-09-14T22:53:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40719 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334788+0000", 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334788+0000", 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334788+0000", 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334791+0000", 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4609999999999998e-06 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334797+0000", 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1920000000000002e-06 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334802+0000", 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0089999999999998e-06 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.343737+0000", 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089346730000000006 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.343809+0000", 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2525000000000001e-05 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.343881+0000", 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2286000000000006e-05 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40720 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1783808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.334809+0000", 2024-09-14T22:53:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.334568054, 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0093001100000000003, 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40720 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334809+0000", 2024-09-14T22:53:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334809+0000", 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334809+0000", 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2599999999999998e-07 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334811+0000", 2024-09-14T22:53:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0480000000000001e-06 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334817+0000", 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1489999999999999e-06 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334822+0000", 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8720000000000001e-06 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.343942+0000", 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091197750000000001 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.344028+0000", 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6540999999999997e-05 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.344109+0000", 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0298999999999998e-05 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40721 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 1832960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.334828+0000", 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.33454815999999998, 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0094445480000000005, 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40721 2024-09-14T22:53:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334828+0000", 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334828+0000", 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334829+0000", 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9999999999999998e-07 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334831+0000", 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9259999999999999e-06 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334837+0000", 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0730000000000001e-06 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.334842+0000", 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4729999999999999e-06 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.344138+0000", 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0092960270000000001 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.344210+0000", 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1095999999999998e-05 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.344273+0000", 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3453000000000004e-05 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40927 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.341253+0000", 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.32812331, 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0092036949999999996, 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40927 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341253+0000", 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341253+0000", 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341254+0000", 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341256+0000", 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.871e-06 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341262+0000", 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0020000000000003e-06 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341266+0000", 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9479999999999999e-06 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.350289+0000", 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090220730000000002 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.350369+0000", 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0532999999999996e-05 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.350457+0000", 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7891000000000003e-05 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:40928 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.341272+0000", 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.32810478999999998, 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0093099069999999992, 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 40928 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341272+0000", 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341272+0000", 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341272+0000", 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3699999999999999e-07 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341273+0000", 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3170000000000001e-06 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341278+0000", 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8829999999999997e-06 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.341283+0000", 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.476e-06 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.350483+0000", 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0092006459999999998 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.350521+0000", 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7404999999999998e-05 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.350582+0000", 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0943000000000002e-05 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:22.759 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:22.760 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:22.761 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:22.758+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:22.861 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:22.860+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:22.864 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:22.864 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:22.864 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:22.864 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:22.873 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:22.962 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:22.960+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:22.967 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:22.968 INFO:teuthology.orchestra.run.smithi076.stdout:639 trunc from 0xeb79b5c to 0xd3a3c07 2024-09-14T22:53:23.043 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:23.043 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:23.052 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:23.063 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:23.061+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:23.161 INFO:teuthology.orchestra.run.smithi076.stdout:640 trunc from 0xd3a3c07 to 0x56ea935 2024-09-14T22:53:23.164 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:23.162+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:23.219 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43191 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1552384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.521968+0000", 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69489011700000003, 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045063573000000003, 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43191 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.521968+0000", 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.521968+0000", 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522192+0000", 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000224126 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522197+0000", 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.865e-06 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522207+0000", 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522213+0000", 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5830000000000003e-06 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.566866+0000", 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044653009 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.566943+0000", 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6194999999999998e-05 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.567032+0000", 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9152000000000003e-05 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43219 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1699840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.522992+0000", 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69386602200000003, 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045069896999999998, 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.224 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43219 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522992+0000", 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522992+0000", 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522993+0000", 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6400000000000001e-07 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522997+0000", 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8410000000000002e-06 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523005+0000", 2024-09-14T22:53:23.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523011+0000", 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3679999999999998e-06 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.567910+0000", 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044898181000000002 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.567982+0000", 2024-09-14T22:53:23.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2268999999999995e-05 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568062+0000", 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0455000000000001e-05 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43230 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1724416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523019+0000", 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69383950599999999, 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045215783000000002, 2024-09-14T22:53:23.228 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43230 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523019+0000", 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523019+0000", 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523087+0000", 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8439e-05 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523091+0000", 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.286e-06 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523098+0000", 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3780000000000003e-06 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523104+0000", 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568089+0000", 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044984718999999999 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568155+0000", 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5950000000000004e-05 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568235+0000", 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9881000000000003e-05 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43231 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1748992~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.232 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523114+0000", 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69374474799999997, 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045295242999999999, 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43231 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523114+0000", 2024-09-14T22:53:23.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523114+0000", 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523575+0000", 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00046139999999999999 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523578+0000", 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.768e-06 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523587+0000", 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0219999999999993e-06 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523593+0000", 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0059999999999998e-06 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568262+0000", 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044668753999999998 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568329+0000", 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7742000000000003e-05 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568409+0000", 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9550999999999998e-05 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43232 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1773568~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523602+0000", 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69325647499999998, 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045025359000000001, 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43232 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523602+0000", 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523602+0000", 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523602+0000", 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9499999999999998e-07 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523605+0000", 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.21e-06 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523611+0000", 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4300000000000003e-06 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523616+0000", 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2009999999999998e-06 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568451+0000", 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044834369999999998 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568523+0000", 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2574e-05 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568627+0000", 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010417900000000001 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43233 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1798144~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523623+0000", 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69323511299999996, 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045205196000000003, 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43233 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523623+0000", 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523623+0000", 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523757+0000", 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013384599999999999 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523764+0000", 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.311e-06 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523773+0000", 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.191e-06 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523778+0000", 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6640000000000003e-06 2024-09-14T22:53:23.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568658+0000", 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044880041000000002 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568727+0000", 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9066999999999994e-05 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568828+0000", 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010107600000000001 2024-09-14T22:53:23.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43234 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1822720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523788+0000", 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69307062699999999, 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045264944000000001, 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.246 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43234 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523788+0000", 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523788+0000", 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523815+0000", 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7319000000000001e-05 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523818+0000", 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.123e-06 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523825+0000", 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9380000000000003e-06 2024-09-14T22:53:23.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523831+0000", 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1600000000000003e-06 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568859+0000", 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045027833000000003 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568941+0000", 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1843999999999998e-05 2024-09-14T22:53:23.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569053+0000", 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111727 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43246 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1847296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523840+0000", 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.693018774, 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045400677, 2024-09-14T22:53:23.250 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43246 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523840+0000", 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523840+0000", 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524155+0000", 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000315385 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524159+0000", 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9280000000000003e-06 2024-09-14T22:53:23.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524168+0000", 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7879999999999995e-06 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524174+0000", 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569083+0000", 2024-09-14T22:53:23.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044909398000000003 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569166+0000", 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2348000000000005e-05 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569240+0000", 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4466000000000002e-05 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43247 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1871872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524213+0000", 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69264487399999997, 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045214745000000001, 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43247 2024-09-14T22:53:23.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524213+0000", 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524213+0000", 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524214+0000", 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524216+0000", 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.767e-06 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524221+0000", 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2259999999999996e-06 2024-09-14T22:53:23.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524226+0000", 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2170000000000002e-06 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569278+0000", 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045051391000000003 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569350+0000", 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2624000000000001e-05 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569428+0000", 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8070000000000003e-05 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43248 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1896448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.259 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524233+0000", 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69262510099999997, 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045386424000000002, 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43248 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524233+0000", 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524233+0000", 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524800+0000", 2024-09-14T22:53:23.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00056699599999999995 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524805+0000", 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7280000000000003e-06 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524813+0000", 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5010000000000004e-06 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524826+0000", 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2459999999999999e-05 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569458+0000", 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044631813999999999 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569522+0000", 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4276999999999995e-05 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569620+0000", 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7647999999999997e-05 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43249 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1921024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524838+0000", 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69201995599999999, 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044971650000000002, 2024-09-14T22:53:23.264 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43249 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524838+0000", 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524838+0000", 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524839+0000", 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524847+0000", 2024-09-14T22:53:23.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6539999999999999e-06 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524859+0000", 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2461e-05 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524869+0000", 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7259999999999997e-06 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569651+0000", 2024-09-14T22:53:23.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044782582000000001 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569737+0000", 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5857000000000001e-05 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569810+0000", 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2872000000000003e-05 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43250 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1945600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524878+0000", 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69198071500000002, 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045111694000000001, 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43250 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.269 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524878+0000", 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524878+0000", 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524878+0000", 2024-09-14T22:53:23.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9500000000000003e-07 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524881+0000", 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.463e-06 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524892+0000", 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1392e-05 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524897+0000", 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569839+0000", 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044942032999999999 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569907+0000", 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7488000000000002e-05 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569989+0000", 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2427000000000002e-05 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43262 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1970176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524906+0000", 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69195260000000003, 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045250118999999998, 2024-09-14T22:53:23.273 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43262 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524906+0000", 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524906+0000", 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524992+0000", 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6601999999999995e-05 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524996+0000", 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2899999999999998e-06 2024-09-14T22:53:23.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525002+0000", 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9170000000000003e-06 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525006+0000", 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1699999999999999e-06 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570025+0000", 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045019075999999998 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570092+0000", 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7670999999999996e-05 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570156+0000", 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3392999999999994e-05 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.277 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43263 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1994752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.525014+0000", 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69184462999999996, 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045422183999999997, 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43263 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525014+0000", 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525014+0000", 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525946+0000", 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00093211200000000002 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525950+0000", 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.296e-06 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525962+0000", 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.155e-05 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525977+0000", 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5726000000000002e-05 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570286+0000", 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044308301000000001 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570367+0000", 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1094000000000007e-05 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570436+0000", 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9104999999999997e-05 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43266 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 2068480~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.282 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.526045+0000", 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69081315300000001, 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044990645000000003, 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43266 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526045+0000", 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526045+0000", 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526046+0000", 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4200000000000001e-07 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526048+0000", 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4600000000000002e-06 2024-09-14T22:53:23.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526057+0000", 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9809999999999992e-06 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526062+0000", 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2179999999999998e-06 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570857+0000", 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044794816000000001 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570953+0000", 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5762999999999998e-05 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.571036+0000", 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2965000000000003e-05 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.286 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43267 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 2093056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.526069+0000", 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.69078927199999995, 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045143575999999998, 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43267 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526069+0000", 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526069+0000", 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.527922+0000", 2024-09-14T22:53:23.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0018524329999999999 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.527931+0000", 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4579999999999999e-06 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.527944+0000", 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.527952+0000", 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8189999999999992e-06 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.571075+0000", 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043123203999999998 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.571146+0000", 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1073000000000002e-05 2024-09-14T22:53:23.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.571213+0000", 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6784000000000003e-05 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43342 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1947648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.530273+0000", 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.68658528799999996, 2024-09-14T22:53:23.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045144380999999997, 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43342 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.530273+0000", 2024-09-14T22:53:23.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.530273+0000", 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.531922+0000", 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001648942 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.531927+0000", 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7979999999999996e-06 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.531941+0000", 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4596999999999999e-05 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.531949+0000", 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4259999999999997e-06 2024-09-14T22:53:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.575253+0000", 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043303696000000003 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.575328+0000", 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5724999999999997e-05 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.575417+0000", 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9197000000000007e-05 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43466 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.536111+0000", 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.68074722499999996, 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045088069000000001, 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.296 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43466 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536111+0000", 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536111+0000", 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536111+0000", 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2100000000000001e-07 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536112+0000", 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0699999999999999e-06 2024-09-14T22:53:23.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536115+0000", 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.881e-06 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536118+0000", 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581079+0000", 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044961360999999998 2024-09-14T22:53:23.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581124+0000", 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5048e-05 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581199+0000", 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4946999999999999e-05 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.300 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43467 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.536136+0000", 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.68072234399999998, 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045225284999999997, 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43467 2024-09-14T22:53:23.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536136+0000", 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536136+0000", 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536136+0000", 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536138+0000", 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.567e-06 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536143+0000", 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3639999999999998e-06 2024-09-14T22:53:23.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536147+0000", 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0749999999999999e-06 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581233+0000", 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045085393000000001 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581298+0000", 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4908999999999998e-05 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581361+0000", 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3584999999999995e-05 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43468 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.305 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.536153+0000", 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.68070488299999998, 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045398223000000001, 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43468 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536153+0000", 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536153+0000", 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.537919+0000", 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017654649999999999 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.537924+0000", 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7480000000000001e-06 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.537934+0000", 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0549e-05 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.537944+0000", 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4409999999999999e-06 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581404+0000", 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043460038999999999 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581478+0000", 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4153999999999995e-05 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581552+0000", 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3826999999999996e-05 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.309 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.310 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.310 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:23.312 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:23.312 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:23.309+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:641 write 0xd5b5ebd thru 0xd5bd29c (0x73e0 bytes) 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:642 trunc from 0xd5bd29d to 0x7ac6e4e 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:643 read 0x4594b92 thru 0x4596461 (0x18d0 bytes) 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:644 punch from 0x33f9643 to 0x33f9f0e, (0x8cb bytes) 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:645 punch from 0x371d399 to 0x37276e3, (0xa34a bytes) 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:646 read 0x34c80e7 thru 0x34cf441 (0x735b bytes) 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:647 punch from 0x6895190 to 0x689a72b, (0x559b bytes) 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:648 write 0x8a097da thru 0x8a1970e (0xff35 bytes) 2024-09-14T22:53:23.313 INFO:teuthology.orchestra.run.smithi076.stdout:649 writesame 0x35c3044 thru 0x35c5751 data_size 0x1(0x270e bytes) 2024-09-14T22:53:23.339 INFO:teuthology.orchestra.run.smithi076.stdout:650 read 0x733f081 thru 0x7347fe2 (0x8f62 bytes) 2024-09-14T22:53:23.345 INFO:teuthology.orchestra.run.smithi076.stdout:651 trunc from 0x8a1970f to 0x386ebd3 2024-09-14T22:53:23.413 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:23.411+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:23.505 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:23.506 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:23.513 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:23.512+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:23.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:23.614 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:23.613+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:23.618 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:23.618 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:23.618 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:23.618 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:23.618 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:23.626 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:23.715 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:23.714+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:23.732 INFO:teuthology.orchestra.run.smithi076.stdout:652 read 0xcb005 thru 0xd63bf (0xb3bb bytes) 2024-09-14T22:53:23.744 INFO:teuthology.orchestra.run.smithi076.stdout:654 trunc from 0x386ebd3 to 0xcdfd08d 2024-09-14T22:53:23.751 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:23.753 INFO:teuthology.orchestra.run.smithi076.stdout:656 trunc from 0xcdfd08d to 0xae954ed 2024-09-14T22:53:23.759 INFO:teuthology.orchestra.run.smithi076.stdout:657 write 0x25b27e1 thru 0x25b5977 (0x3197 bytes) 2024-09-14T22:53:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:658 punch from 0x3fb2206 to 0x3fb60a5, (0x3e9f bytes) 2024-09-14T22:53:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:659 read 0xa34da26 thru 0xa35c209 (0xe7e4 bytes) 2024-09-14T22:53:23.762 INFO:teuthology.orchestra.run.smithi076.stdout:660 write 0xab0dbc6 thru 0xab175cb (0x9a06 bytes) 2024-09-14T22:53:23.765 INFO:teuthology.orchestra.run.smithi076.stdout:661 write 0x7eb64f0 thru 0x7ec03b9 (0x9eca bytes) 2024-09-14T22:53:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:23.769 INFO:teuthology.orchestra.run.smithi076.stdout:664 punch from 0x906f68b to 0x907d753, (0xe0c8 bytes) 2024-09-14T22:53:23.770 INFO:teuthology.orchestra.run.smithi076.stdout:665 trunc from 0xae954ed to 0xa1507c6 2024-09-14T22:53:23.780 INFO:teuthology.orchestra.run.smithi076.stdout:666 trunc from 0xa1507c6 to 0xa963b90 2024-09-14T22:53:23.786 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:23.786 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35862 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1644544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081674+0000", 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7026014730000001, 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878086000000001, 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.787 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35862 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081674+0000", 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081674+0000", 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081715+0000", 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0847999999999999e-05 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081718+0000", 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5030000000000002e-06 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081727+0000", 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9420000000000001e-06 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081733+0000", 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.049e-06 2024-09-14T22:53:23.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130376+0000", 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048642699999999997 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130455+0000", 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9054999999999994e-05 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130552+0000", 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6989000000000003e-05 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.7025337199999999, 2024-09-14T22:53:23.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:53:23.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:53:23.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:23.797 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.702315824, 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:53:23.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35913 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1832960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084589+0000", 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.699686421, 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048859330999999999, 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35913 2024-09-14T22:53:23.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084589+0000", 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084589+0000", 2024-09-14T22:53:23.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084590+0000", 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7399999999999998e-07 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084592+0000", 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.074e-06 2024-09-14T22:53:23.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084597+0000", 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6439999999999997e-06 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084602+0000", 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5000000000000001e-06 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133277+0000", 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048675527000000003 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133356+0000", 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8788000000000005e-05 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133448+0000", 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2324000000000006e-05 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6996321779999999, 2024-09-14T22:53:23.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:23.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:53:23.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:53:23.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6994737629999999, 2024-09-14T22:53:23.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:23.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:53:23.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:23.818 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6994464969999998, 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.823 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6992393560000001, 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.824 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:53:23.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.828 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6992050920000001, 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:53:23.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:53:23.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.697573046, 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:53:23.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.839 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.697548434, 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:23.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:53:23.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.697414148, 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.845 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.850 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.69636444, 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:53:23.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:53:23.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.855 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6962622550000002, 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.856 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:53:23.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6962383509999999, 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:53:23.861 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:23.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:53:23.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:23.866 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.696204831, 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:53:23.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:53:23.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.696050185, 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.872 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:23.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:53:23.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.877 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6960090110000001, 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:53:23.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:23.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6959808509999998, 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.883 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:53:23.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:53:23.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:23.888 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.6957914600000001, 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:23.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:53:23.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:23.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:23.893 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:23.894 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:53:23.895 INFO:teuthology.orchestra.run.smithi076.stdout:669 writesame 0xb8ac80d thru 0xb8ace7c data_size 0x8(0x670 bytes) 2024-09-14T22:53:23.895 INFO:teuthology.orchestra.run.smithi076.stdout:670 trunc from 0xb8ace7d to 0x1912d0e 2024-09-14T22:53:23.895 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:23.893+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:23.896 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:23.896 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:23.996 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:23.994+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:24.061 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:64130 3.7 e38/24 3:e0c199cd:::rbd_data.1135c0c4962c.0000000000000001:head v 38'45, mlcod=38'43)", 2024-09-14T22:53:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.014506956999999999, 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014544019, 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045691+0000", 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054194+0000", 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085033850000000005 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054219+0000", 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4533999999999999e-05 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054231+0000", 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2617e-05 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054328+0000", 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6978000000000006e-05 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054355+0000", 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6302e-05 2024-09-14T22:53:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:64132 3.14 e38/24 3:28028895:::rbd_data.1135c0c4962c.0000000000000002:head v 38'81, mlcod=38'79)", 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.053115+0000", 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0070814939999999998, 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007233382, 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053115+0000", 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053115+0000", 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053117+0000", 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3710000000000001e-06 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054369+0000", 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0012522830000000001 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054378+0000", 2024-09-14T22:53:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0459999999999994e-06 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054383+0000", 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054429+0000", 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6121000000000001e-05 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054448+0000", 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9352999999999999e-05 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 2 2024-09-14T22:53:24.070 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:24.076 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:24.096 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:24.095+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:24.101 INFO:teuthology.orchestra.run.smithi076.stdout:671 punch from 0x2a7107 to 0x2b4aa8, (0xd9a1 bytes) 2024-09-14T22:53:24.102 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:24.102 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:24.111 INFO:teuthology.orchestra.run.smithi076.stdout:672 write 0x4ca9fae thru 0x4cb35ae (0x9601 bytes) 2024-09-14T22:53:24.115 INFO:teuthology.orchestra.run.smithi076.stdout:674 punch from 0x23910d7 to 0x23982b4, (0x71dd bytes) 2024-09-14T22:53:24.122 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:24.122 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:24.126 INFO:teuthology.orchestra.run.smithi076.stdout:675 write 0x9c64760 thru 0x9c7262b (0xdecc bytes) 2024-09-14T22:53:24.129 INFO:teuthology.orchestra.run.smithi076.stdout:677 read 0x8b0e23b thru 0x8b157f6 (0x75bc bytes) 2024-09-14T22:53:24.140 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:24.140 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:679 writesame 0xe8051b6 thru 0xe814ae8 data_size 0x5(0xf933 bytes) 2024-09-14T22:53:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:681 punch from 0xc51a72f to 0xc51c590, (0x1e61 bytes) 2024-09-14T22:53:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:24.141 INFO:teuthology.orchestra.run.smithi076.stdout:685 write 0xc3d3a50 thru 0xc3d753d (0x3aee bytes) 2024-09-14T22:53:24.157 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:24.157 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:24.158 INFO:teuthology.orchestra.run.smithi076.stdout:686 trunc from 0xe814ae9 to 0x17772d6 2024-09-14T22:53:24.176 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:24.176 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:24.178 INFO:teuthology.orchestra.run.smithi076.stdout:691 read 0x12b1d61 thru 0x12bdc98 (0xbf38 bytes) 2024-09-14T22:53:24.178 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:24.179 INFO:teuthology.orchestra.run.smithi076.stdout:693 punch from 0x10e6c23 to 0x10ed12b, (0x6508 bytes) 2024-09-14T22:53:24.179 INFO:teuthology.orchestra.run.smithi076.stdout:696 read 0x147f9f3 thru 0x148aec4 (0xb4d2 bytes) 2024-09-14T22:53:24.187 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:24.187 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:24.187 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:24.187 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:24.187 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:24.188 INFO:teuthology.orchestra.run.smithi076.stdout:697 write 0x3eff269 thru 0x3f05008 (0x5da0 bytes) 2024-09-14T22:53:24.195 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:24.195 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:24.196 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:24.197 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:24.195+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:24.199 INFO:teuthology.orchestra.run.smithi076.stdout:698 trunc from 0x3f05009 to 0x1a5afc9 2024-09-14T22:53:24.214 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:24.214 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:24.214 INFO:teuthology.orchestra.run.smithi076.stdout:699 read 0x1473ba2 thru 0x147b15f (0x75be bytes) 2024-09-14T22:53:24.214 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:24.231 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:24.231 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:24.234 INFO:teuthology.orchestra.run.smithi076.stdout:701 write 0xc8d24af thru 0xc8dffaa (0xdafc bytes) 2024-09-14T22:53:24.236 INFO:teuthology.orchestra.run.smithi076.stdout:703 writesame 0x7eaeca8 thru 0x7eb4e4e data_size 0xd(0x61a7 bytes) 2024-09-14T22:53:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:704 punch from 0x32d35f9 to 0x32de2da, (0xace1 bytes) 2024-09-14T22:53:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:707 read 0x34d46e3 thru 0x34d8290 (0x3bae bytes) 2024-09-14T22:53:24.241 INFO:teuthology.orchestra.run.smithi076.stdout:708 write 0xc7bc092 thru 0xc7c7992 (0xb901 bytes) 2024-09-14T22:53:24.245 INFO:teuthology.orchestra.run.smithi076.stdout:710 read 0x471f731 thru 0x4728f54 (0x9824 bytes) 2024-09-14T22:53:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:711 read 0xb831f8 thru 0xb874cd (0x42d6 bytes) 2024-09-14T22:53:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:713 read 0x60043ed thru 0x6004b55 (0x769 bytes) 2024-09-14T22:53:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:24.246 INFO:teuthology.orchestra.run.smithi076.stdout:715 write 0x94ca68c thru 0x94d5eeb (0xb860 bytes) 2024-09-14T22:53:24.251 INFO:teuthology.orchestra.run.smithi076.stdout:716 read 0x6f4e2d7 thru 0x6f4e522 (0x24c bytes) 2024-09-14T22:53:24.251 INFO:teuthology.orchestra.run.smithi076.stdout:717 read 0x151c4ac thru 0x1521356 (0x4eab bytes) 2024-09-14T22:53:24.258 INFO:teuthology.orchestra.run.smithi076.stdout:718 write 0xd964599 thru 0xd97422f (0xfc97 bytes) 2024-09-14T22:53:24.262 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:24.262 INFO:teuthology.orchestra.run.smithi076.stdout:720 writesame 0x10a2dd5 thru 0x10a7201 data_size 0x1(0x442d bytes) 2024-09-14T22:53:24.273 INFO:teuthology.orchestra.run.smithi076.stdout:721 write 0x3927bdf thru 0x393319b (0xb5bd bytes) 2024-09-14T22:53:24.277 INFO:teuthology.orchestra.run.smithi076.stdout:722 clone 8 order 18 su 65536 sc 13 2024-09-14T22:53:24.297 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:24.296+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:24.307 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:24.307 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:24.316 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:24.400 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:24.397+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:24.419 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:24.419 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:24.427 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:24.499 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:24.498+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:24.530 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:24.530 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:24.539 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:24.623 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:24.622+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:24.682 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:24.682 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:24.691 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:24.732 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:24.731+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:24.801 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:24.800+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:24.834 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:53:24.839 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:24.839 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:24.849 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:24.902 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:24.900+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:24.915 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone7 from 0xd974230 (overlap 0x17772d6) to 0x4b16f7 2024-09-14T22:53:24.999 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:25.000 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:25.003 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:25.001+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:25.009 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:25.017 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T22:53:25.105 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:25.104+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:25.185 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:25.186 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:25.199 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:25.239 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:25.238+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:25.306 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:25.304+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:25.407 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:25.405+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:25.437 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:25.437 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:25.449 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:25.507 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:25.506+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:25.596 INFO:teuthology.orchestra.run.smithi076.stdout:723 trunc from 0xd974230 to 0x97f9eb2 2024-09-14T22:53:25.600 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:25.600 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:25.607 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:25.606+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:25.610 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:25.622 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:25.622 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:25.623 INFO:teuthology.orchestra.run.smithi076.stdout:727 punch from 0x170f26f to 0x1718c28, (0x99b9 bytes) 2024-09-14T22:53:25.623 INFO:teuthology.orchestra.run.smithi076.stdout:728 punch from 0x6a109c7 to 0x6a10a4c, (0x85 bytes) 2024-09-14T22:53:25.629 INFO:teuthology.orchestra.run.smithi076.stdout:729 trunc from 0x97f9eb2 to 0x3fbf67a 2024-09-14T22:53:25.656 INFO:teuthology.orchestra.run.smithi076.stdout:730 trunc from 0x3fbf67a to 0x1afacef 2024-09-14T22:53:25.683 INFO:teuthology.orchestra.run.smithi076.stdout:731 read 0xeabf thru 0x12852 (0x3d94 bytes) 2024-09-14T22:53:25.689 INFO:teuthology.orchestra.run.smithi076.stdout:732 trunc from 0x1afacef to 0x5d5109c 2024-09-14T22:53:25.703 INFO:teuthology.orchestra.run.smithi076.stdout:735 write 0x96368aa thru 0x9637f5e (0x16b5 bytes) 2024-09-14T22:53:25.708 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:25.707+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:25.718 INFO:teuthology.orchestra.run.smithi076.stdout:736 write 0xe7cefa3 thru 0xe7d9485 (0xa4e3 bytes) 2024-09-14T22:53:25.721 INFO:teuthology.orchestra.run.smithi076.stdout:738 write 0x601b965 thru 0x602217c (0x6818 bytes) 2024-09-14T22:53:25.728 INFO:teuthology.orchestra.run.smithi076.stdout:740 read 0xafd3619 thru 0xafe198c (0xe374 bytes) 2024-09-14T22:53:25.744 INFO:teuthology.orchestra.run.smithi076.stdout:741 trunc from 0xe7d9486 to 0x141d8da 2024-09-14T22:53:25.746 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:25.746 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:25.757 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:25.772 INFO:teuthology.orchestra.run.smithi076.stdout:743 read 0x62a527 thru 0x632e51 (0x892b bytes) 2024-09-14T22:53:25.789 INFO:teuthology.orchestra.run.smithi076.stdout:744 write 0xa85f2f2 thru 0xa860433 (0x1142 bytes) 2024-09-14T22:53:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:745 punch from 0x73763bc to 0x7379625, (0x3269 bytes) 2024-09-14T22:53:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:747 read 0x31bc02 thru 0x31f6a6 (0x3aa5 bytes) 2024-09-14T22:53:25.796 INFO:teuthology.orchestra.run.smithi076.stdout:748 punch from 0xba4c26 to 0xbac9c3, (0x7d9d bytes) 2024-09-14T22:53:25.797 INFO:teuthology.orchestra.run.smithi076.stdout:749 writesame 0x7ecda5 thru 0x7f3f68 data_size 0x12(0x71c4 bytes) 2024-09-14T22:53:25.802 INFO:teuthology.orchestra.run.smithi076.stdout:750 write 0x596ce5d thru 0x597567b (0x881f bytes) 2024-09-14T22:53:25.809 INFO:teuthology.orchestra.run.smithi076.stdout:751 writesame 0x515ef1f thru 0x516aabe data_size 0x26(0xbba0 bytes) 2024-09-14T22:53:25.809 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:25.808+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:25.813 INFO:teuthology.orchestra.run.smithi076.stdout:752 read 0x1182200 thru 0x118b872 (0x9673 bytes) 2024-09-14T22:53:25.813 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:25.818 INFO:teuthology.orchestra.run.smithi076.stdout:756 trunc from 0xa860434 to 0xe95d6a4 2024-09-14T22:53:25.821 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:25.821 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:25.821 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:53:25.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:53:25.824 INFO:teuthology.orchestra.run.smithi076.stdout:757 read 0xb45d2e3 thru 0xb462a5e (0x577c bytes) 2024-09-14T22:53:25.824 INFO:teuthology.orchestra.run.smithi076.stdout:758 punch from 0x29ab29a to 0x29b3348, (0x80ae bytes) 2024-09-14T22:53:25.829 INFO:teuthology.orchestra.run.smithi076.stdout:760 trunc from 0xe95d6a4 to 0xa2ed5ce 2024-09-14T22:53:25.839 INFO:teuthology.orchestra.run.smithi076.stdout:761 read 0x425e65e thru 0x425f31c (0xcbf bytes) 2024-09-14T22:53:25.839 INFO:teuthology.orchestra.run.smithi076.stdout:762 clone 9 order 24 su 262144 sc 3 2024-09-14T22:53:25.886 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:25.886 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:25.895 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:25.936 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:25.935+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:26.011 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:26.009+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:26.040 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:26.040 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:26.049 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:26.111 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:26.111 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 39, 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:53:24.818536+0000", 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5804032, 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "nodeep-scrub", 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:53:26.112 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:53:26.113 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:26.114 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:53:26.115 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:26.116 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:53:26.117 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:26.118 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:26.119 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:53:26.120 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.121 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:26.122 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:26.123 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:26.124 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:26.125 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.126 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.127 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 2.9900000095367432, 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 2.9900000095367432, 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.128 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:26.129 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:26.130 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:26.130 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:53:26.130 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:53:26.130 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:53:26.130 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:53:26.130 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:53:26.130 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:26.130 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "39", 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:26.131 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 10, 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 39, 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:26.132 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.133 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:26.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.5, 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.5, 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.5, 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.5, 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 1, 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 1, 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 1 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:26.135 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:26.136 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.137 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:53:26.139 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.140 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:26.142 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.143 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.144 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.145 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:53:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:53:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 19, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:53:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:53:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:53:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:53:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:53:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 37, 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:53:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:53:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 25, 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:53:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "2.6", 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: 5, 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:53:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [], 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [], 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:53:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [], 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:53:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:53:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:26.196 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:26.196 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:26.110+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:26.198 INFO:tasks.thrashosds.thrasher:j is {'epoch': 39, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:53:24.818536+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 10, 'snap_epoch': 39, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [5, 2]}, {'pgid': '3.0', 'osds': [6, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:53:26.198 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-14T22:53:26.198 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:26.198 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:26.202 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:26.243 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:26.244 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:26.242+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:26.307 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:26.308 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:26.313 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:26.312+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:26.316 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:26.363 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:26.364 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:26.380 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:26.380 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:26.396 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:26.396 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:26.411 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:26.412 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:26.414 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:26.413+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:26.419 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:26.419 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:26.428 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:26.428 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:26.429 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:26.444 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:26.445 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:26.461 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:26.461 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:26.477 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:26.477 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:26.529 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:26.528+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:26.572 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:26.572 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53585 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.412719+0000", 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1576248609999999, 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015640300999999999, 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.573 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53585 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412719+0000", 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412719+0000", 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412720+0000", 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5600000000000001e-07 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412721+0000", 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.658e-06 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412727+0000", 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5720000000000002e-06 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412731+0000", 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6439999999999999e-06 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428208+0000", 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015476894 2024-09-14T22:53:26.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428279+0000", 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0877000000000006e-05 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428360+0000", 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.03e-05 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53586 3.3 3:c14366e7:::rbd_data.11354ec77855.0000000000000010:6 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.412737+0000", 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1576067449999998, 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01579879, 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53586 2024-09-14T22:53:26.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412737+0000", 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412737+0000", 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412738+0000", 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9799999999999999e-07 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412739+0000", 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.778e-06 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412745+0000", 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4609999999999999e-06 2024-09-14T22:53:26.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412749+0000", 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.527e-06 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428388+0000", 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015638378000000001 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428455+0000", 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7352000000000002e-05 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428536+0000", 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0995999999999995e-05 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53587 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.412869+0000", 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.157474809, 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015718623000000001, 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53587 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412869+0000", 2024-09-14T22:53:26.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412869+0000", 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412870+0000", 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2e-07 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412873+0000", 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7190000000000001e-06 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412881+0000", 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1489999999999994e-06 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412886+0000", 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6690000000000003e-06 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428578+0000", 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015691396 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428588+0000", 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0169999999999999e-05 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53588 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.412895+0000", 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1574494770000001, 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015724919, 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.587 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53588 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412895+0000", 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412895+0000", 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412895+0000", 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1899999999999998e-07 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412906+0000", 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0988000000000001e-05 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412912+0000", 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7780000000000004e-06 2024-09-14T22:53:26.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.412924+0000", 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1861e-05 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428612+0000", 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015688357999999999 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428620+0000", 2024-09-14T22:53:26.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.515e-06 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53589 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413011+0000", 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1573326769999999, 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015638181000000001, 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.591 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53589 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413011+0000", 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413011+0000", 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413012+0000", 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.06e-07 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413015+0000", 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8059999999999999e-06 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413022+0000", 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0849999999999999e-06 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413028+0000", 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7799999999999997e-06 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428642+0000", 2024-09-14T22:53:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015614714999999999 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428650+0000", 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1890000000000001e-06 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53590 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413036+0000", 2024-09-14T22:53:26.595 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1573081049999998, 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015642396999999999, 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53590 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413036+0000", 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413036+0000", 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413036+0000", 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.08e-07 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413038+0000", 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.874e-06 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413044+0000", 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0769999999999996e-06 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413049+0000", 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8690000000000003e-06 2024-09-14T22:53:26.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428671+0000", 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015621512000000001 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428678+0000", 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6569999999999997e-06 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.599 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53591 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413056+0000", 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1572884069999998, 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015651459, 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53591 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413056+0000", 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413056+0000", 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413056+0000", 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413058+0000", 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8080000000000001e-06 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413063+0000", 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5459999999999999e-06 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413068+0000", 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5360000000000003e-06 2024-09-14T22:53:26.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428700+0000", 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015632521999999999 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428707+0000", 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6719999999999998e-06 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53592 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413074+0000", 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1572700559999998, 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015666050000000001, 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53592 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.604 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413074+0000", 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413074+0000", 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413074+0000", 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4299999999999999e-07 2024-09-14T22:53:26.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413076+0000", 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8220000000000001e-06 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413082+0000", 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2900000000000002e-06 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413086+0000", 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7690000000000004e-06 2024-09-14T22:53:26.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428733+0000", 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015646891 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428740+0000", 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9349999999999997e-06 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.607 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53596 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413255+0000", 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.157089466, 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015608754000000001, 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53596 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413255+0000", 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413255+0000", 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413255+0000", 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7799999999999997e-07 2024-09-14T22:53:26.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413257+0000", 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.852e-06 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413262+0000", 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5979999999999996e-06 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413267+0000", 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7809999999999996e-06 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428856+0000", 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015589298 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428863+0000", 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9469999999999997e-06 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.611 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53597 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413273+0000", 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1570713050000001, 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015621164, 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53597 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413273+0000", 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413273+0000", 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413273+0000", 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.46e-07 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413275+0000", 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6330000000000001e-06 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413280+0000", 2024-09-14T22:53:26.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5260000000000001e-06 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413285+0000", 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5879999999999999e-06 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428886+0000", 2024-09-14T22:53:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015601128000000001 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428894+0000", 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.943e-06 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53598 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413290+0000", 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1570537220000001, 2024-09-14T22:53:26.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015638532, 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53598 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413290+0000", 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413290+0000", 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413291+0000", 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1699999999999999e-07 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413292+0000", 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7179999999999999e-06 2024-09-14T22:53:26.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413298+0000", 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5990000000000001e-06 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413303+0000", 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6059999999999996e-06 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428922+0000", 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015619337 2024-09-14T22:53:26.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428929+0000", 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9550000000000003e-06 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53599 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413308+0000", 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1570360380000002, 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015650223000000001, 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.621 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53599 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413308+0000", 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413308+0000", 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413308+0000", 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413310+0000", 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.779e-06 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413316+0000", 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8409999999999996e-06 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413321+0000", 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.763e-06 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428951+0000", 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015629774999999999 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428958+0000", 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6879999999999999e-06 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53601 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413326+0000", 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1570179920000001, 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015660132, 2024-09-14T22:53:26.625 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53601 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413326+0000", 2024-09-14T22:53:26.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413326+0000", 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413326+0000", 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6399999999999998e-07 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413328+0000", 2024-09-14T22:53:26.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.689e-06 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413334+0000", 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.716e-06 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413338+0000", 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.313e-06 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428980+0000", 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015641407 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.428986+0000", 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6429999999999998e-06 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53602 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413350+0000", 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1569937289999999, 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015665466999999999, 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53602 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.630 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413350+0000", 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413350+0000", 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413351+0000", 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413353+0000", 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4499999999999998e-06 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413360+0000", 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3860000000000004e-06 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413364+0000", 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7790000000000003e-06 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.429009+0000", 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015644333 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.429016+0000", 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.063e-06 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53604 3.3 3.e766c283 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.413371+0000", 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1569730530000002, 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015673062000000001, 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.634 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53604 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413371+0000", 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413371+0000", 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413372+0000", 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413373+0000", 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.823e-06 2024-09-14T22:53:26.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413379+0000", 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6849999999999998e-06 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.413384+0000", 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6999999999999999e-06 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.429037+0000", 2024-09-14T22:53:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015652836999999999 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.429044+0000", 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5610000000000001e-06 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:59055 3.3 3:c6b09e77:::rbd_header.1135c0c4962c:head [watch reconnect cookie 140460070133904 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.856431+0000", 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.713912863, 2024-09-14T22:53:26.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018060052, 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 59055 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.856431+0000", 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.856431+0000", 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.856432+0000", 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7e-07 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.856433+0000", 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8470000000000001e-06 2024-09-14T22:53:26.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.856440+0000", 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.856445+0000", 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0119999999999996e-06 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.862731+0000", 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062861330000000002 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.862809+0000", 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7216000000000001e-05 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.874491+0000", 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011682593999999999 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64132 3.14 3:28028895:::rbd_data.1135c0c4962c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3276800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.042246+0000", 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.52809806, 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043608325000000003, 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64132 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.042246+0000", 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.042246+0000", 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.042247+0000", 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3900000000000004e-07 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.052348+0000", 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010101017 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.052374+0000", 2024-09-14T22:53:26.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6383000000000001e-05 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.052385+0000", 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0662000000000001e-05 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.052474+0000", 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8591999999999995e-05 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:26.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.052608+0000", 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000134182 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053006+0000", 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000398584 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.072736+0000", 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019729871 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.085728+0000", 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012992063999999999 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.085800+0000", 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1384999999999995e-05 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.085854+0000", 2024-09-14T22:53:26.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64135 3.3 3:c3dcbc87:::rbd_data.1135c0c4962c.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3276800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.052410+0000", 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.5179340740000002, 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034058123000000003, 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64135 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.052410+0000", 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.052410+0000", 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.052411+0000", 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3859999999999999e-06 2024-09-14T22:53:26.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062455+0000", 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010043203000000001 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062478+0000", 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3487999999999999e-05 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062494+0000", 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.56e-05 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062596+0000", 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010219500000000001 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062759+0000", 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016312499999999999 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063323+0000", 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00056434499999999995 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.071586+0000", 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0082622549999999996 2024-09-14T22:53:26.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086378+0000", 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014792181999999999 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086431+0000", 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2821999999999997e-05 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086468+0000", 2024-09-14T22:53:26.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7521999999999998e-05 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:64134 3.1d e38/24 3:bf0dbf4f:::rbd_data.1135c0c4962c.0000000000000004:head v 38'53, mlcod=37'50)", 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.058971+0000", 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.5113729129999998, 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.027278205999999999, 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.656 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.058971+0000", 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.058971+0000", 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.058972+0000", 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2600000000000002e-07 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.060789+0000", 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001817199 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.060807+0000", 2024-09-14T22:53:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7510999999999999e-05 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.060816+0000", 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6579999999999997e-06 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.060879+0000", 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3120999999999994e-05 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.060898+0000", 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8723e-05 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086183+0000", 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.025284726 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086249+0000", 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6642000000000003e-05 2024-09-14T22:53:26.660 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:64133 3.f e38/24 3:f670552a:::rbd_data.1135c0c4962c.0000000000000003:head v 38'53, mlcod=35'51)", 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.063634+0000", 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.5067102650000002, 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.022961029000000001, 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:26.661 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063634+0000", 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063634+0000", 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063636+0000", 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.756e-06 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.065896+0000", 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022599679999999998 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.065911+0000", 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5625999999999999e-05 2024-09-14T22:53:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.065928+0000", 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6773e-05 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.065998+0000", 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9536999999999995e-05 2024-09-14T22:53:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.066025+0000", 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7923e-05 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086552+0000", 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.020527059 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086595+0000", 2024-09-14T22:53:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2386999999999999e-05 2024-09-14T22:53:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:26.668 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:26.668 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:26.666+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:26.769 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:26.767+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:26.771 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:26.772 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:26.780 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:26.849 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:53:26.893 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:26.892+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:26.927 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:26.927 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:26.927 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:26.927 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:26.928 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:26.936 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:26.970 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:26.969+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:26.974 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone8 from 0xa2ed5ce (overlap 0x141d8da) to 0x202f49 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43191 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1552384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.521968+0000", 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5468669540000004, 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045063573000000003, 2024-09-14T22:53:27.071 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43191 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.521968+0000", 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.521968+0000", 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522192+0000", 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000224126 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522197+0000", 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.865e-06 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522207+0000", 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522213+0000", 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5830000000000003e-06 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.566866+0000", 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044653009 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.566943+0000", 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6194999999999998e-05 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.567032+0000", 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9152000000000003e-05 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.075 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43219 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1699840~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.522992+0000", 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5458428590000004, 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045069896999999998, 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.076 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43219 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522992+0000", 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522992+0000", 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522993+0000", 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6400000000000001e-07 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.522997+0000", 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8410000000000002e-06 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523005+0000", 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9189999999999999e-06 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523011+0000", 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3679999999999998e-06 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.567910+0000", 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044898181000000002 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.567982+0000", 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2268999999999995e-05 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568062+0000", 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0455000000000001e-05 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.080 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43230 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1724416~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523019+0000", 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5458163430000003, 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045215783000000002, 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43230 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523019+0000", 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523019+0000", 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523087+0000", 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8439e-05 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523091+0000", 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.286e-06 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523098+0000", 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3780000000000003e-06 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523104+0000", 2024-09-14T22:53:27.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568089+0000", 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044984718999999999 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568155+0000", 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5950000000000004e-05 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568235+0000", 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9881000000000003e-05 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43231 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1748992~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523114+0000", 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5457215849999999, 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045295242999999999, 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.085 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43231 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523114+0000", 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523114+0000", 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523575+0000", 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00046139999999999999 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523578+0000", 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.768e-06 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523587+0000", 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0219999999999993e-06 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523593+0000", 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0059999999999998e-06 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568262+0000", 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044668753999999998 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568329+0000", 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7742000000000003e-05 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568409+0000", 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9550999999999998e-05 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.089 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43232 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1773568~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523602+0000", 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5452333119999997, 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045025359000000001, 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43232 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523602+0000", 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523602+0000", 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523602+0000", 2024-09-14T22:53:27.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9499999999999998e-07 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523605+0000", 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.21e-06 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523611+0000", 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4300000000000003e-06 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523616+0000", 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2009999999999998e-06 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568451+0000", 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044834369999999998 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568523+0000", 2024-09-14T22:53:27.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2574e-05 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568627+0000", 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010417900000000001 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43233 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1798144~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523623+0000", 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5452119499999997, 2024-09-14T22:53:27.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045205196000000003, 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43233 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523623+0000", 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523623+0000", 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523757+0000", 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013384599999999999 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523764+0000", 2024-09-14T22:53:27.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.311e-06 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523773+0000", 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.191e-06 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523778+0000", 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6640000000000003e-06 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568658+0000", 2024-09-14T22:53:27.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044880041000000002 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568727+0000", 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9066999999999994e-05 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568828+0000", 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010107600000000001 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43234 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1822720~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523788+0000", 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5450474639999996, 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045264944000000001, 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43234 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523788+0000", 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523788+0000", 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523815+0000", 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7319000000000001e-05 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523818+0000", 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.123e-06 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523825+0000", 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9380000000000003e-06 2024-09-14T22:53:27.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523831+0000", 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1600000000000003e-06 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568859+0000", 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045027833000000003 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.568941+0000", 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1843999999999998e-05 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569053+0000", 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111727 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43246 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1847296~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.103 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.523840+0000", 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.544995611, 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045400677, 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43246 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523840+0000", 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.523840+0000", 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524155+0000", 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000315385 2024-09-14T22:53:27.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524159+0000", 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9280000000000003e-06 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524168+0000", 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7879999999999995e-06 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524174+0000", 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569083+0000", 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044909398000000003 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569166+0000", 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2348000000000005e-05 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569240+0000", 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4466000000000002e-05 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43247 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1871872~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524213+0000", 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5446217109999996, 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045214745000000001, 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.108 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43247 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524213+0000", 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524213+0000", 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524214+0000", 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524216+0000", 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.767e-06 2024-09-14T22:53:27.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524221+0000", 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2259999999999996e-06 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524226+0000", 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2170000000000002e-06 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569278+0000", 2024-09-14T22:53:27.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045051391000000003 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569350+0000", 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2624000000000001e-05 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569428+0000", 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8070000000000003e-05 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43248 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1896448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524233+0000", 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5446019379999996, 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045386424000000002, 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43248 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524233+0000", 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524233+0000", 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524800+0000", 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00056699599999999995 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524805+0000", 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7280000000000003e-06 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524813+0000", 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5010000000000004e-06 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524826+0000", 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2459999999999999e-05 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569458+0000", 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044631813999999999 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569522+0000", 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4276999999999995e-05 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569620+0000", 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7647999999999997e-05 2024-09-14T22:53:27.116 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43249 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1921024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524838+0000", 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5439967929999998, 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044971650000000002, 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43249 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524838+0000", 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524838+0000", 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524839+0000", 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524847+0000", 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6539999999999999e-06 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524859+0000", 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2461e-05 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524869+0000", 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7259999999999997e-06 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569651+0000", 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044782582000000001 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569737+0000", 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5857000000000001e-05 2024-09-14T22:53:27.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569810+0000", 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2872000000000003e-05 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43250 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1945600~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524878+0000", 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5439575520000002, 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045111694000000001, 2024-09-14T22:53:27.121 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43250 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524878+0000", 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524878+0000", 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524878+0000", 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9500000000000003e-07 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524881+0000", 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.463e-06 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524892+0000", 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1392e-05 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524897+0000", 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569839+0000", 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044942032999999999 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569907+0000", 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7488000000000002e-05 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.569989+0000", 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2427000000000002e-05 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43262 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1970176~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.125 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.524906+0000", 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5439294370000001, 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045250118999999998, 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43262 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524906+0000", 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524906+0000", 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524992+0000", 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6601999999999995e-05 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.524996+0000", 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2899999999999998e-06 2024-09-14T22:53:27.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525002+0000", 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9170000000000003e-06 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525006+0000", 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1699999999999999e-06 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570025+0000", 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045019075999999998 2024-09-14T22:53:27.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570092+0000", 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7670999999999996e-05 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570156+0000", 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3392999999999994e-05 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.129 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43263 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1994752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.525014+0000", 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5438214669999999, 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045422183999999997, 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43263 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525014+0000", 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525014+0000", 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525946+0000", 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00093211200000000002 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525950+0000", 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.296e-06 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525962+0000", 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.155e-05 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.525977+0000", 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5726000000000002e-05 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570286+0000", 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044308301000000001 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570367+0000", 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1094000000000007e-05 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570436+0000", 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9104999999999997e-05 2024-09-14T22:53:27.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43266 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 2068480~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.526045+0000", 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5427899900000002, 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044990645000000003, 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.134 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43266 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526045+0000", 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526045+0000", 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526046+0000", 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4200000000000001e-07 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526048+0000", 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4600000000000002e-06 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526057+0000", 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9809999999999992e-06 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526062+0000", 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2179999999999998e-06 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570857+0000", 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044794816000000001 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.570953+0000", 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5762999999999998e-05 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.571036+0000", 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2965000000000003e-05 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43267 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 2093056~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.526069+0000", 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5427661090000004, 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045143575999999998, 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43267 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526069+0000", 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.526069+0000", 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.527922+0000", 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0018524329999999999 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.527931+0000", 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4579999999999999e-06 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.527944+0000", 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2805e-05 2024-09-14T22:53:27.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.527952+0000", 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8189999999999992e-06 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.571075+0000", 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043123203999999998 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.571146+0000", 2024-09-14T22:53:27.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1073000000000002e-05 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.571213+0000", 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6784000000000003e-05 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43342 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1947648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.530273+0000", 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5385621250000003, 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045144380999999997, 2024-09-14T22:53:27.142 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43342 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.530273+0000", 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.530273+0000", 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.531922+0000", 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001648942 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.531927+0000", 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7979999999999996e-06 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.531941+0000", 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4596999999999999e-05 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.531949+0000", 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4259999999999997e-06 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.575253+0000", 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043303696000000003 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.575328+0000", 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5724999999999997e-05 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.575417+0000", 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9197000000000007e-05 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43466 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 1998848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.536111+0000", 2024-09-14T22:53:27.146 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5327240619999998, 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045088069000000001, 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43466 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536111+0000", 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536111+0000", 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536111+0000", 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2100000000000001e-07 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536112+0000", 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0699999999999999e-06 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536115+0000", 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.881e-06 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536118+0000", 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5409999999999999e-06 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581079+0000", 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044961360999999998 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581124+0000", 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5048e-05 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581199+0000", 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4946999999999999e-05 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43467 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 2023424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.536136+0000", 2024-09-14T22:53:27.150 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5326991809999999, 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045225284999999997, 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43467 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536136+0000", 2024-09-14T22:53:27.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536136+0000", 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536136+0000", 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9200000000000002e-07 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536138+0000", 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.567e-06 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536143+0000", 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3639999999999998e-06 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536147+0000", 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0749999999999999e-06 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581233+0000", 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045085393000000001 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581298+0000", 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4908999999999998e-05 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581361+0000", 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3584999999999995e-05 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:43468 3.c 3:3350eb25:::rbd_data.11354ec77855.0000000000000022:6 [read 2048000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.536153+0000", 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5326817200000002, 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.045398223000000001, 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 43468 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536153+0000", 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.536153+0000", 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.537919+0000", 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017654649999999999 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.537924+0000", 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7480000000000001e-06 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.537934+0000", 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0549e-05 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.537944+0000", 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4409999999999999e-06 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581404+0000", 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.043460038999999999 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581478+0000", 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4153999999999995e-05 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.581552+0000", 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3826999999999996e-05 2024-09-14T22:53:27.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.159 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.159 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.159 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.159 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.159 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:27.160 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T22:53:27.160 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:27.161 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:27.159+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:27.183 INFO:teuthology.orchestra.run.smithi076.stdout:763 write 0xc3ab1fb thru 0xc3ba045 (0xee4b bytes) 2024-09-14T22:53:27.187 INFO:teuthology.orchestra.run.smithi076.stdout:764 read 0x46f7476 thru 0x4701ecd (0xaa58 bytes) 2024-09-14T22:53:27.188 INFO:teuthology.orchestra.run.smithi076.stdout:765 write 0xac4c4eb thru 0xac5a931 (0xe447 bytes) 2024-09-14T22:53:27.192 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:768 read 0x932cc82 thru 0x933b47e (0xe7fd bytes) 2024-09-14T22:53:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:27.193 INFO:teuthology.orchestra.run.smithi076.stdout:773 write 0x7e5c527 thru 0x7e61df6 (0x58d0 bytes) 2024-09-14T22:53:27.196 INFO:teuthology.orchestra.run.smithi076.stdout:774 punch from 0x9229aa to 0x9314f8, (0xeb4e bytes) 2024-09-14T22:53:27.196 INFO:teuthology.orchestra.run.smithi076.stdout:776 write 0x31fd1c thru 0x323af2 (0x3dd7 bytes) 2024-09-14T22:53:27.228 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:27.228 INFO:teuthology.orchestra.run.smithi076.stdout:779 read 0x2d5be6c thru 0x2d63205 (0x739a bytes) 2024-09-14T22:53:27.228 INFO:teuthology.orchestra.run.smithi076.stdout:780 writesame 0xb4d6efc thru 0xb4dbd73 data_size 0x24(0x4e78 bytes) 2024-09-14T22:53:27.231 INFO:teuthology.orchestra.run.smithi076.stdout:781 clone 10 order 19 su 4096 sc 2 2024-09-14T22:53:27.260 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:27.259+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:27.271 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:27.271 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:27.279 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:27.363 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:27.362+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:27.423 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:27.423 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:27.423 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:27.423 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:27.423 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:27.431 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:27.462 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:27.461+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:27.563 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:27.562+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:27.576 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:27.576 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:27.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:27.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:27.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.576 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4924695689999998, 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.577 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:53:27.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:53:27.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.581 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4922516730000002, 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:53:27.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:53:27.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4895680269999998, 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.586 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:53:27.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4894096120000002, 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.590 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:27.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4893823460000002, 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:53:27.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:53:27.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.598 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4891752050000004, 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4891409409999996, 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:53:27.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4875088950000004, 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.607 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4874842829999997, 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:53:27.611 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:53:27.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:53:27.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4873499969999999, 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:53:27.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:53:27.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:53:27.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4863002889999999, 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:53:27.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:53:27.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4861981039999996, 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:53:27.624 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.628 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4861741999999998, 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:27.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:53:27.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:53:27.632 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4861406800000001, 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4859860339999997, 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:27.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:53:27.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:53:27.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.48594486, 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4859166999999998, 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:27.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.4857273089999996, 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:53:27.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.5696374820000001, 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.5696176509999997, 2024-09-14T22:53:27.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:27.662 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:27.663 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:27.664 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:27.664 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:27.662+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:27.764 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:27.763+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:27.808 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:27.808 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:27.817 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:27.836 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:53:27.893 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:27.892+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:27.926 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:27.926 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:27.926 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:27.926 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:27.926 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:27.935 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:27.940 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone9 from 0xc3ba046 (overlap 0xa2ed5ce) to 0x104aefb 2024-09-14T22:53:27.980 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:27.978+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.547989606999998, 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:53:28.058 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:28.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:53:28.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5811 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.670752722, 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035921055, 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.066 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5811 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5199999999999998e-07 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385023+0000", 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0035e-05 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385029+0000", 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5250000000000002e-06 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385035+0000", 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2460000000000003e-06 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420764+0000", 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035729819000000003 2024-09-14T22:53:28.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420840+0000", 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5784000000000005e-05 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420934+0000", 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3294000000000005e-05 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.070 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5864 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.669056754, 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035943627999999998, 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5864 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386920+0000", 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021137200000000001 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386923+0000", 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4230000000000001e-06 2024-09-14T22:53:28.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386932+0000", 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4290000000000001e-06 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386938+0000", 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0839999999999997e-06 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422480+0000", 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035542705000000001 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422559+0000", 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8385000000000003e-05 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422652+0000", 2024-09-14T22:53:28.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.323e-05 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5865 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 1046528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.668818266000001, 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035838531, 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5865 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386950+0000", 2024-09-14T22:53:28.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2749999999999998e-06 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386956+0000", 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2090000000000002e-06 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386961+0000", 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1959999999999999e-06 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422683+0000", 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035721387 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422727+0000", 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.401e-05 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422785+0000", 2024-09-14T22:53:28.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8950000000000003e-05 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53826 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 913408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6371311510000002, 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035814618999999999, 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.081 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53826 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418635+0000", 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4300000000000003e-07 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418637+0000", 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0090000000000002e-06 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418643+0000", 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1109999999999996e-06 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418648+0000", 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.152e-06 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454259+0000", 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035610615999999998 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454326+0000", 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7565999999999997e-05 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454449+0000", 2024-09-14T22:53:28.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012262199999999999 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53830 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6371114479999997, 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036000748999999999, 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53830 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:28.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999996e-07 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418656+0000", 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0729999999999999e-06 2024-09-14T22:53:28.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418663+0000", 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418668+0000", 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1100000000000002e-06 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454478+0000", 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035809797999999997 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454544+0000", 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6089999999999999e-05 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454655+0000", 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110788 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53831 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.637090959, 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036273573000000003, 2024-09-14T22:53:28.091 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53831 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418675+0000", 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418677+0000", 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.086e-06 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418684+0000", 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.754e-06 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418689+0000", 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6300000000000003e-06 2024-09-14T22:53:28.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454701+0000", 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036011950000000001 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454806+0000", 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104558 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454948+0000", 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142177 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53833 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 915456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6370681329999996, 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.03648067, 2024-09-14T22:53:28.096 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53833 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418698+0000", 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418700+0000", 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.091e-06 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418706+0000", 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6850000000000004e-06 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418712+0000", 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2059999999999998e-06 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454991+0000", 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036279441000000003 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455065+0000", 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3617999999999999e-05 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455178+0000", 2024-09-14T22:53:28.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000113205 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53860 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 862208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6370470790000002, 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036670083999999999, 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.101 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53860 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418719+0000", 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6899999999999997e-07 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418725+0000", 2024-09-14T22:53:28.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1569999999999997e-06 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418733+0000", 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9009999999999994e-06 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418738+0000", 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.553e-06 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455206+0000", 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036467558999999997 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455273+0000", 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7378000000000004e-05 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455388+0000", 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114967 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54365 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 737280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6240344679999996, 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035967086000000002, 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54365 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431733+0000", 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9230000000000001e-06 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431742+0000", 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5669999999999995e-06 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431747+0000", 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.383e-06 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467514+0000", 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035766569999999998 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467582+0000", 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8373000000000004e-05 2024-09-14T22:53:28.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467698+0000", 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000115864 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54366 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.111 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6240112809999996, 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036163224000000001, 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54366 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.112 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6199999999999998e-07 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431757+0000", 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.272e-06 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431764+0000", 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8240000000000002e-06 2024-09-14T22:53:28.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431769+0000", 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1410000000000004e-06 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467728+0000", 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035959526999999998 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467797+0000", 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8681000000000002e-05 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467917+0000", 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012031699999999999 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54367 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6239903179999997, 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036385946000000002, 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54367 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:28.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431777+0000", 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9290000000000001e-06 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431784+0000", 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3790000000000003e-06 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431789+0000", 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1660000000000002e-06 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467947+0000", 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036158542000000002 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468020+0000", 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2429999999999996e-05 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468161+0000", 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014109499999999999 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54368 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6239688750000001, 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036591052999999998, 2024-09-14T22:53:28.122 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54368 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:28.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431797+0000", 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431803+0000", 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.438e-06 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431812+0000", 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5520000000000005e-06 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431817+0000", 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5389999999999998e-06 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468191+0000", 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036374191 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468265+0000", 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3385000000000003e-05 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468387+0000", 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012255400000000001 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54369 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.623940524, 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036784182999999998, 2024-09-14T22:53:28.127 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54369 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4200000000000001e-07 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431827+0000", 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.221e-06 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431834+0000", 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5400000000000001e-06 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431839+0000", 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7570000000000004e-06 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468417+0000", 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036577927000000003 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468489+0000", 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2412999999999999e-05 2024-09-14T22:53:28.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468609+0000", 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119883 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54370 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 765952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:28.132 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6239207000000002, 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036997090000000003, 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54370 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2100000000000002e-07 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431847+0000", 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9750000000000001e-06 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431853+0000", 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.173e-06 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431858+0000", 2024-09-14T22:53:28.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8810000000000004e-06 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468641+0000", 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036783320000000001 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468718+0000", 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6417999999999999e-05 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468842+0000", 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012390200000000001 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54371 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:28.137 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6239013839999998, 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037201153000000001, 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54371 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2300000000000002e-07 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431866+0000", 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0700000000000001e-06 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431875+0000", 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.225e-06 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431880+0000", 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1189999999999996e-06 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468871+0000", 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036991375999999999 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468948+0000", 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6991999999999997e-05 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469065+0000", 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011694800000000001 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54373 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6238790549999997, 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037377190999999997, 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54373 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:28.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431887+0000", 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431889+0000", 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0470000000000001e-06 2024-09-14T22:53:28.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431895+0000", 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4760000000000003e-06 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431900+0000", 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8539999999999996e-06 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469091+0000", 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037190894000000002 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469151+0000", 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9867999999999997e-05 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469263+0000", 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011267700000000001 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64133 3.f 3:f670552a:::rbd_data.1135c0c4962c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3276800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.012533586, 2024-09-14T22:53:28.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044079376000000003, 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64133 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043233+0000", 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.638e-06 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062793+0000", 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019559246999999998 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062810+0000", 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7431e-05 2024-09-14T22:53:28.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062819+0000", 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.329e-06 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062890+0000", 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0605999999999995e-05 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063027+0000", 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000136606 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063504+0000", 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000477875 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086724+0000", 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023220082 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087212+0000", 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048734800000000002 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087262+0000", 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0652999999999998e-05 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087311+0000", 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8560999999999998e-05 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:64130 3.7 e38/24 3:e0c199cd:::rbd_data.1135c0c4962c.0000000000000001:head v 38'45, mlcod=38'43)", 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.0100752999999996, 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.040529201000000001, 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.155 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045691+0000", 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054194+0000", 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085033850000000005 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054219+0000", 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4533999999999999e-05 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054231+0000", 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2617e-05 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054328+0000", 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6978000000000006e-05 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054355+0000", 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6302e-05 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086122+0000", 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.031767008999999999 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086219+0000", 2024-09-14T22:53:28.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7361999999999994e-05 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:76917 3.15 3:ab9de9be:::rbd_header.113535cec067:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.1779001650000001, 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047974171000000003, 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:28.160 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 76917 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877866+0000", 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7899999999999998e-07 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877869+0000", 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4429999999999999e-06 2024-09-14T22:53:28.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877875+0000", 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5160000000000002e-06 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877879+0000", 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.206e-06 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:28.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877906+0000", 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7254999999999999e-05 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877949+0000", 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2830000000000002e-05 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.925839+0000", 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047890342000000002 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.164 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:28.165 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:28.165 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:28.166 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T22:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:784 punch from 0xa36ed46 to 0xa374aff, (0x5db9 bytes) 2024-09-14T22:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:785 write 0xbef4e2b thru 0xbefb450 (0x6626 bytes) 2024-09-14T22:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:786 write 0x44902b4 thru 0x4496cc1 (0x6a0e bytes) 2024-09-14T22:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:787 punch from 0x8f8e6b6 to 0x8f9a1df, (0xbb29 bytes) 2024-09-14T22:53:28.167 INFO:teuthology.orchestra.run.smithi076.stdout:788 clone 11 order 24 su 1048576 sc 2 2024-09-14T22:53:28.167 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:28.165+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:28.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:28.267 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:28.265+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:28.271 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:28.271 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:28.271 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:28.272 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:28.280 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:28.367 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:28.366+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:28.382 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:28.383 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:28.383 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:28.383 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:28.383 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:28.391 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:28.475 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:28.474+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:28.489 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:28.538 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:28.551 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:28.551 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:28.551 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:28.551 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.551 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6859819720000004, 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.552 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.553 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.554 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:53:28.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6850443820000001, 2024-09-14T22:53:28.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:53:28.560 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6849530189999999, 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.561 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:53:28.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.566 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.566 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:28.566 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6843081340000001, 2024-09-14T22:53:28.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:53:28.567 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.567 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.567 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.567 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.567 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.567 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.568 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:53:28.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.568 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.568 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:28.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:28.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.570 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:53:28.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:53:28.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:53:28.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:53:28.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.572 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.572 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.572 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:53:28.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:53:28.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:53:28.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:53:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:53:28.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:53:28.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:53:28.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:53:28.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.576 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.576 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.576 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:53:28.576 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:53:28.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.577 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.577 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.577 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.577 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.577 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.578 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:28.578 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6838207550000002, 2024-09-14T22:53:28.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:53:28.578 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.578 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.579 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.579 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.579 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.579 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.579 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:53:28.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.580 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.580 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:28.581 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.581 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.581 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.581 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:28.582 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.582 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.582 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:28.583 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:53:28.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.583 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.583 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.583 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:53:28.584 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:53:28.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.584 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.584 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.584 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:53:28.585 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:53:28.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.585 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.585 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.585 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:53:28.586 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:53:28.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.586 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.586 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.586 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:53:28.586 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:53:28.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.587 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.587 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.587 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:53:28.587 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:53:28.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.588 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.588 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.588 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:53:28.588 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:53:28.589 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.589 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.589 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.589 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.589 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.590 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.590 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:28.590 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.68293348, 2024-09-14T22:53:28.590 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:53:28.590 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.591 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.591 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.591 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.591 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.591 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.592 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:53:28.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.592 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.592 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.592 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.592 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:28.593 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.593 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.593 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.593 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:28.594 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.594 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.594 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.594 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:53:28.595 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:53:28.595 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.595 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.595 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.595 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:53:28.596 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:53:28.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.596 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.596 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.596 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:53:28.597 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:53:28.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.597 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.597 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.597 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:53:28.598 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:53:28.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.598 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.598 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.598 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:53:28.599 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:53:28.599 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.599 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.599 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.599 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:53:28.600 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:53:28.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.600 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.600 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:53:28.601 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:53:28.601 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.601 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.601 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.602 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.602 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:28.602 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6823410560000003, 2024-09-14T22:53:28.602 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:53:28.603 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.603 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.603 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.603 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.603 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.604 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.604 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:53:28.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.604 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.604 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.605 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.605 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:28.605 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:28.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.607 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.607 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:28.607 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:53:28.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.607 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.608 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:53:28.608 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:53:28.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.608 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.609 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:53:28.609 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:53:28.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.609 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.610 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:53:28.610 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:53:28.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.610 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.611 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:53:28.611 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:53:28.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.611 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.611 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.612 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:53:28.612 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:53:28.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.612 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.612 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.613 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:53:28.613 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:53:28.613 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.613 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.613 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.614 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.614 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:28.614 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6816082400000001, 2024-09-14T22:53:28.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:53:28.615 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.615 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.615 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.615 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.616 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.616 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.616 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:53:28.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.616 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.617 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.617 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:28.617 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.618 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.618 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:28.618 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.619 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.619 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:53:28.619 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:53:28.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.620 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.620 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:53:28.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:53:28.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.621 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.621 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:53:28.621 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:53:28.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.621 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.622 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.622 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:53:28.622 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:53:28.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.622 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.623 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:53:28.623 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:53:28.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.623 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.624 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:53:28.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:53:28.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.624 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.624 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:53:28.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:53:28.625 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.625 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.625 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6815090909999997, 2024-09-14T22:53:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:53:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.626 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:28.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.630 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6811906329999999, 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:53:28.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6806811389999998, 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:53:28.635 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:53:28.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.639 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6806232799999998, 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.640 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:53:28.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.644 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6798306570000001, 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:53:28.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.648 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.679155218, 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.649 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:53:28.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:53:28.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.653 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6790219080000002, 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.654 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:28.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:53:28.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6784049850000002, 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:53:28.658 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:53:28.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.678338128, 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:53:28.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6778873169999997, 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.668 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:53:28.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:53:28.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6777670840000001, 2024-09-14T22:53:28.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:28.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:53:28.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6764489569999999, 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:28.677 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:28.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:28.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:28.681 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:28.682 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:28.683 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:28.684 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:28.684 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:28.684 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:28.684 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:28.685 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:28.685 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:28.685 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:28.686 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:28.686 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:28.686 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:28.686 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:28.681+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:28.693 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:28.693 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:28.710 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:28.710 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:28.727 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:28.727 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:28.784 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:28.783+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:28.799 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:28.799 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:28.808 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:28.893 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:28.892+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:28.942 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone10 from 0xc3ba046 (overlap 0xc3ba046) to 0x0 2024-09-14T22:53:28.971 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:28.971 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:28.982 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:29.024 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:29.023+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:29.086 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:29.084+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:29.162 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:29.162 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:29.174 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:29.215 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:29.214+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:29.287 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:29.286+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:29.352 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:29.352 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:29.364 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:29.388 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:29.386+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:29.488 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:29.487+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:29.498 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:29.498 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:29.509 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:29.590 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:29.588+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:29.690 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:29.689+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:29.696 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:29.696 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:29.707 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:29.792 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:29.791+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:29.852 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:53:29.878 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:29.879 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:29.888 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:29.891 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:29.890+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:29.992 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:29.991+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:30.049 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:30.049 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:30.060 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:30.093 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:30.092+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:30.194 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:30.193+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:30.221 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:30.221 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:30.230 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:30.295 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:30.294+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:30.390 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:30.390 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:30.396 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:30.395+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:30.401 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:30.497 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:30.496+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:30.585 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:30.585 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:30.596 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:30.637 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:30.636+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:30.699 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:30.697+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:30.738 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:30.772 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:30.773 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:30.783 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:30.826 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:30.824+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:30.851 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:53:30.879 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:30.897 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:30.898 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:30.900 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:30.899+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:30.914 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:30.914 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:30.932 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:30.932 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:30.951 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:30.951 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:30.967 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:30.967 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:30.969 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:30.969 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:30.978 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:30.987 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:31.001 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:31.000+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:31.005 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:31.005 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:31.102 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:31.101+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:31.174 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:31.174 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:31.174 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:31.174 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:31.174 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:31.185 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:31.199 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:31.199 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:31.199 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.5237425767857853 2024-09-14T22:53:31.200 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 0.5237425767857853 2024-09-14T22:53:31.229 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:31.228+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:31.303 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:31.301+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103971 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 3903488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.268063+0000", 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.079134768999999994, 2024-09-14T22:53:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049632979000000001, 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103971 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268063+0000", 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268063+0000", 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268064+0000", 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7199999999999999e-07 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268066+0000", 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0930000000000001e-06 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268072+0000", 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268077+0000", 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6099999999999999e-06 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.317606+0000", 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049528824999999999 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.317646+0000", 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0442999999999999e-05 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.317696+0000", 2024-09-14T22:53:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0436999999999998e-05 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103972 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 3911680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.268083+0000", 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.079114895000000005, 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049837339000000001, 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103972 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268083+0000", 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268083+0000", 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268084+0000", 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000002e-07 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268086+0000", 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2290000000000002e-06 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268092+0000", 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7679999999999997e-06 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268096+0000", 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6260000000000003e-06 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.317727+0000", 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049630707000000003 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.317807+0000", 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9921999999999998e-05 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.317921+0000", 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011369 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103973 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 3919872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.268103+0000", 2024-09-14T22:53:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.079095713999999998, 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049966692, 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103973 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268103+0000", 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268103+0000", 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268103+0000", 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3599999999999999e-07 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268105+0000", 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1090000000000001e-06 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268111+0000", 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6030000000000004e-06 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268115+0000", 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6770000000000003e-06 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.317954+0000", 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049838753999999999 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.318013+0000", 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8906999999999997e-05 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.318069+0000", 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6205999999999997e-05 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103974 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 3928064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.268121+0000", 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.079076795000000005, 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050070563999999998, 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103974 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268121+0000", 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268121+0000", 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268122+0000", 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7800000000000002e-07 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268124+0000", 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9759999999999998e-06 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268130+0000", 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.251e-06 2024-09-14T22:53:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268136+0000", 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0050000000000001e-06 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.318100+0000", 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049963699 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.318140+0000", 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0218e-05 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.318192+0000", 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2036999999999997e-05 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103975 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.268142+0000", 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.079056187999999999, 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050273393999999999, 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103975 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268142+0000", 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.268142+0000", 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.269922+0000", 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001779957 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.269927+0000", 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0499999999999999e-06 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.269939+0000", 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1568e-05 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.269946+0000", 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0820000000000001e-06 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.318223+0000", 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048277103000000002 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.318307+0000", 2024-09-14T22:53:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4648000000000007e-05 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.318415+0000", 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010798600000000001 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103984 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4009984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.270124+0000", 2024-09-14T22:53:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.077074537999999998, 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049731465000000002, 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103984 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270124+0000", 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270124+0000", 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270124+0000", 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5900000000000003e-07 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270126+0000", 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7129999999999999e-06 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270131+0000", 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2299999999999999e-06 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270136+0000", 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:53:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.319743+0000", 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049607322000000002 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.319805+0000", 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1581000000000006e-05 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.319855+0000", 2024-09-14T22:53:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0571000000000001e-05 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103985 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.270142+0000", 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.077056545000000004, 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049858094999999998, 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103985 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270142+0000", 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270142+0000", 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270142+0000", 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9499999999999998e-07 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270144+0000", 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9319999999999999e-06 2024-09-14T22:53:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270150+0000", 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7400000000000001e-06 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270154+0000", 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6369999999999998e-06 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.319880+0000", 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049725327999999999 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.319930+0000", 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9864000000000002e-05 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.320000+0000", 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0198999999999997e-05 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103986 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.270161+0000", 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.077037707999999996, 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050063045, 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103986 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270161+0000", 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.270161+0000", 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271577+0000", 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014166129999999999 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271581+0000", 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9790000000000004e-06 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271592+0000", 2024-09-14T22:53:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0638e-05 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271600+0000", 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9529999999999999e-06 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.320033+0000", 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048433450000000003 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.320123+0000", 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0122000000000002e-05 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.320224+0000", 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010029 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103993 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4083712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.271768+0000", 2024-09-14T22:53:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.075430663999999994, 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04968645, 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103993 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271768+0000", 2024-09-14T22:53:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271768+0000", 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271768+0000", 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271770+0000", 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3489999999999999e-06 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271777+0000", 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6259999999999997e-06 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271782+0000", 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.006e-06 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321328+0000", 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049545836000000003 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321368+0000", 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0448999999999998e-05 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321454+0000", 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5734000000000003e-05 2024-09-14T22:53:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103994 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4091904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.271789+0000", 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.075409017999999994, 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049857960999999999, 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103994 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271789+0000", 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271789+0000", 2024-09-14T22:53:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271824+0000", 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4717e-05 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271827+0000", 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4470000000000002e-06 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271835+0000", 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6190000000000002e-06 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271841+0000", 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7189999999999998e-06 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321488+0000", 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049647717000000001 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321577+0000", 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8318000000000004e-05 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321647+0000", 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0424000000000002e-05 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103995 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4100096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.271849+0000", 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.075348961000000006, 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049913051, 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103995 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271849+0000", 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271849+0000", 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271930+0000", 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0409999999999998e-05 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271933+0000", 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8830000000000002e-06 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271942+0000", 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4110000000000002e-06 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271948+0000", 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2659999999999998e-06 2024-09-14T22:53:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321674+0000", 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049725448999999998 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321714+0000", 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0609999999999999e-05 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321762+0000", 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8022000000000002e-05 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:103996 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4108288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.271958+0000", 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.075240024000000003, 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049982305999999997, 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 103996 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271958+0000", 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.271958+0000", 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.273919+0000", 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019604380000000001 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.273923+0000", 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7679999999999999e-06 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.273932+0000", 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6810000000000004e-06 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.273939+0000", 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4799999999999998e-06 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321789+0000", 2024-09-14T22:53:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047850436000000003 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321843+0000", 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4373999999999998e-05 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.321941+0000", 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7128999999999998e-05 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:104118 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4550656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.282112+0000", 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.065086565999999998, 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04982872, 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 104118 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.282112+0000", 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.282112+0000", 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.283921+0000", 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0018089429999999999 2024-09-14T22:53:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.283925+0000", 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7589999999999997e-06 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.283936+0000", 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0859999999999999e-05 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.283944+0000", 2024-09-14T22:53:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5310000000000004e-06 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.331734+0000", 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047790531999999997 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.331818+0000", 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3746999999999996e-05 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.331940+0000", 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012234799999999999 2024-09-14T22:53:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:104127 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4624384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.284144+0000", 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.063054416000000002, 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049741092000000001, 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 104127 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284144+0000", 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284144+0000", 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284144+0000", 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2500000000000001e-07 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284145+0000", 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2860000000000001e-06 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284149+0000", 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1190000000000001e-06 2024-09-14T22:53:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284151+0000", 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7199999999999998e-06 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.333708+0000", 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049556463000000002 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.333787+0000", 2024-09-14T22:53:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9076e-05 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.333885+0000", 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8103000000000006e-05 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:104128 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4632576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.284154+0000", 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.063043802999999995, 2024-09-14T22:53:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049944144000000003, 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 104128 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284154+0000", 2024-09-14T22:53:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284154+0000", 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284155+0000", 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2499999999999999e-07 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284156+0000", 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2219999999999999e-06 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284159+0000", 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.529e-06 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284162+0000", 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8559999999999998e-06 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.333929+0000", 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049766587000000001 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.334002+0000", 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2868999999999996e-05 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.334099+0000", 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6855999999999996e-05 2024-09-14T22:53:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:104129 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4640768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.284165+0000", 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.063032843000000005, 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050097823999999999, 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 104129 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284165+0000", 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.284165+0000", 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.285919+0000", 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017535820000000001 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.285931+0000", 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1837e-05 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.285941+0000", 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0521e-05 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.285949+0000", 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5510000000000002e-06 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.334137+0000", 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048187894000000002 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.334210+0000", 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3038000000000001e-05 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.334263+0000", 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3400999999999999e-05 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:104210 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 4997120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.293247+0000", 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.053951194000000001, 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049665323999999997, 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 104210 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293247+0000", 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293247+0000", 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293399+0000", 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015211599999999999 2024-09-14T22:53:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293403+0000", 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.664e-06 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293410+0000", 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293415+0000", 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4280000000000004e-06 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.342820+0000", 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049404915000000001 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.342863+0000", 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2747999999999998e-05 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.342912+0000", 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9743999999999997e-05 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:104211 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 5005312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.293429+0000", 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.053769048, 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049675867999999998, 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 104211 2024-09-14T22:53:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293429+0000", 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293429+0000", 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293548+0000", 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118752 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293551+0000", 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2270000000000002e-06 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293557+0000", 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.755e-06 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293561+0000", 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.844e-06 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.342939+0000", 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049378126000000001 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.343011+0000", 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2471999999999993e-05 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.343105+0000", 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3691999999999997e-05 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:104212 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 5013504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.293568+0000", 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.053629939000000001, 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667034999999998, 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 104212 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293568+0000", 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293568+0000", 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293766+0000", 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019803400000000001 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293774+0000", 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9300000000000003e-06 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293782+0000", 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4919999999999996e-06 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.293795+0000", 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3502999999999999e-05 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.343137+0000", 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049341510999999998 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.343183+0000", 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6204e-05 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.343235+0000", 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2361000000000002e-05 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:104214 3.1d 3:bc4f8ee2:::rbd_data.1135aa5cee52.000000000000000b:c [read 5029888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T22:53:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.294045+0000", 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.053153324000000002, 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049745392999999999, 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 104214 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.294045+0000", 2024-09-14T22:53:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.294045+0000", 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.294924+0000", 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00087886799999999999 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.294928+0000", 2024-09-14T22:53:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5639999999999998e-06 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.294939+0000", 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0559e-05 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.294949+0000", 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0558e-05 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.343612+0000", 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048662393999999998 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.343696+0000", 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3665999999999995e-05 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.343790+0000", 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4784000000000006e-05 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:31.441 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:31.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:31.443 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:31.440+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:31.543 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:31.542+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:31.630 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:31.630 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:31.643 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:31.683 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:31.682+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:31.744 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:31.743+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:31.843 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:31.843 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:31.845 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:31.844+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:31.854 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.3 to 0.523743 (8613) 2024-09-14T22:53:31.856 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:31.946 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:31.944+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:32.003 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9198646509999993, 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:53:32.004 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:32.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:53:32.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:53:32.008 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9196467550000005, 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:53:32.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9169631089999992, 2024-09-14T22:53:32.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:53:32.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:53:32.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9168046939999996, 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:32.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:53:32.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9167774279999996, 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.023 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9165702870000008, 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.027 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.031 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9165360230000008, 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:53:32.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9149039769999998, 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.036 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.040 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9148793650000009, 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:53:32.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9147450789999994, 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:53:32.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:53:32.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9136953709999993, 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:53:32.049 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:53:32.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:53:32.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9135931859999999, 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:32.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9135692819999992, 2024-09-14T22:53:32.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:53:32.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9135357620000004, 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.913381116, 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.067 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:53:32.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9133399420000003, 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:53:32.071 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:32.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:53:32.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9133117819999992, 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.076 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:53:32.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:32.080 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.9131223909999999, 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:53:32.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:53:32.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:53:32.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.9970325639999995, 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:53:32.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.997012733, 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.090 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:32.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:53:32.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.095 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:32.097 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:32.097 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:32.095+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:32.197 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:32.196+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:32.242 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:32.242 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:32.242 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:32.242 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:32.250 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:32.298 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:32.297+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:32.392 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:32.393 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:32.393 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:32.393 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:32.393 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:32.399 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:32.397+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:32.401 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:32.501 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:32.499+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:32.544 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.544 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.034356326999998, 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:53:32.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:53:32.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:32.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:53:32.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.552 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5811 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.157119442000001, 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035921055, 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5811 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.553 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5199999999999998e-07 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385023+0000", 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0035e-05 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385029+0000", 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5250000000000002e-06 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385035+0000", 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2460000000000003e-06 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420764+0000", 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035729819000000003 2024-09-14T22:53:32.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420840+0000", 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5784000000000005e-05 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420934+0000", 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3294000000000005e-05 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5864 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.155423473999999, 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035943627999999998, 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.558 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5864 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386920+0000", 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021137200000000001 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386923+0000", 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4230000000000001e-06 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386932+0000", 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4290000000000001e-06 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386938+0000", 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0839999999999997e-06 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422480+0000", 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035542705000000001 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422559+0000", 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8385000000000003e-05 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422652+0000", 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.323e-05 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.562 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5865 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 1046528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.155184986, 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035838531, 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.563 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5865 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386950+0000", 2024-09-14T22:53:32.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2749999999999998e-06 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386956+0000", 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2090000000000002e-06 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386961+0000", 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1959999999999999e-06 2024-09-14T22:53:32.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422683+0000", 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035721387 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422727+0000", 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.401e-05 2024-09-14T22:53:32.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422785+0000", 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8950000000000003e-05 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53826 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 913408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:32.568 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1234978709999996, 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035814618999999999, 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53826 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418635+0000", 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4300000000000003e-07 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418637+0000", 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0090000000000002e-06 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418643+0000", 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1109999999999996e-06 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418648+0000", 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.152e-06 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454259+0000", 2024-09-14T22:53:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035610615999999998 2024-09-14T22:53:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454326+0000", 2024-09-14T22:53:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7565999999999997e-05 2024-09-14T22:53:32.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454449+0000", 2024-09-14T22:53:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012262199999999999 2024-09-14T22:53:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.576 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53830 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1234781680000001, 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036000748999999999, 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53830 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999996e-07 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418656+0000", 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0729999999999999e-06 2024-09-14T22:53:32.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418663+0000", 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418668+0000", 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1100000000000002e-06 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454478+0000", 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035809797999999997 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454544+0000", 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6089999999999999e-05 2024-09-14T22:53:32.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454655+0000", 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110788 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53831 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:32.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1234576789999995, 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036273573000000003, 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53831 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418675+0000", 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418677+0000", 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.086e-06 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418684+0000", 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.754e-06 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418689+0000", 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6300000000000003e-06 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454701+0000", 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036011950000000001 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454806+0000", 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104558 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454948+0000", 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142177 2024-09-14T22:53:32.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53833 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 915456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1234348529999991, 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.03648067, 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.588 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53833 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418698+0000", 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418700+0000", 2024-09-14T22:53:32.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.091e-06 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418706+0000", 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6850000000000004e-06 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418712+0000", 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2059999999999998e-06 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454991+0000", 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036279441000000003 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455065+0000", 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3617999999999999e-05 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455178+0000", 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000113205 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53860 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 862208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1234137989999997, 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036670083999999999, 2024-09-14T22:53:32.593 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53860 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418719+0000", 2024-09-14T22:53:32.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6899999999999997e-07 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418725+0000", 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1569999999999997e-06 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418733+0000", 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9009999999999994e-06 2024-09-14T22:53:32.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418738+0000", 2024-09-14T22:53:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.553e-06 2024-09-14T22:53:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455206+0000", 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036467558999999997 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455273+0000", 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7378000000000004e-05 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455388+0000", 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114967 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54365 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 737280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1104011880000009, 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035967086000000002, 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.599 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54365 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431733+0000", 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9230000000000001e-06 2024-09-14T22:53:32.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431742+0000", 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5669999999999995e-06 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431747+0000", 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.383e-06 2024-09-14T22:53:32.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467514+0000", 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035766569999999998 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467582+0000", 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8373000000000004e-05 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467698+0000", 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000115864 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54366 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1103780010000008, 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036163224000000001, 2024-09-14T22:53:32.604 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54366 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6199999999999998e-07 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431757+0000", 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.272e-06 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431764+0000", 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8240000000000002e-06 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431769+0000", 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1410000000000004e-06 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467728+0000", 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035959526999999998 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467797+0000", 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8681000000000002e-05 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467917+0000", 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012031699999999999 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54367 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1103570380000001, 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036385946000000002, 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.610 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54367 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431777+0000", 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9290000000000001e-06 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431784+0000", 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3790000000000003e-06 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431789+0000", 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1660000000000002e-06 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467947+0000", 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036158542000000002 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468020+0000", 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2429999999999996e-05 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468161+0000", 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014109499999999999 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54368 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1103355950000005, 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036591052999999998, 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54368 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431797+0000", 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431803+0000", 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.438e-06 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431812+0000", 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5520000000000005e-06 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431817+0000", 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5389999999999998e-06 2024-09-14T22:53:32.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468191+0000", 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036374191 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468265+0000", 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3385000000000003e-05 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468387+0000", 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012255400000000001 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54369 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1103072439999995, 2024-09-14T22:53:32.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036784182999999998, 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54369 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4200000000000001e-07 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431827+0000", 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.221e-06 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431834+0000", 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5400000000000001e-06 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431839+0000", 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7570000000000004e-06 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468417+0000", 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036577927000000003 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468489+0000", 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2412999999999999e-05 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468609+0000", 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119883 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54370 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 765952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1102874200000006, 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036997090000000003, 2024-09-14T22:53:32.626 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54370 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2100000000000002e-07 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431847+0000", 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9750000000000001e-06 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431853+0000", 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.173e-06 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431858+0000", 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8810000000000004e-06 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468641+0000", 2024-09-14T22:53:32.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036783320000000001 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468718+0000", 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6417999999999999e-05 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468842+0000", 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012390200000000001 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54371 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1102681039999993, 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037201153000000001, 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54371 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:32.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2300000000000002e-07 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431866+0000", 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0700000000000001e-06 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431875+0000", 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.225e-06 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431880+0000", 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1189999999999996e-06 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468871+0000", 2024-09-14T22:53:32.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036991375999999999 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468948+0000", 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6991999999999997e-05 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469065+0000", 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011694800000000001 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.636 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54373 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.1102457749999992, 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037377190999999997, 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.637 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54373 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431887+0000", 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431889+0000", 2024-09-14T22:53:32.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0470000000000001e-06 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431895+0000", 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4760000000000003e-06 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431900+0000", 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8539999999999996e-06 2024-09-14T22:53:32.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469091+0000", 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037190894000000002 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469151+0000", 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9867999999999997e-05 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469263+0000", 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011267700000000001 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64133 3.f 3:f670552a:::rbd_data.1135c0c4962c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3276800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.4989003059999995, 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044079376000000003, 2024-09-14T22:53:32.642 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64133 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:32.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043233+0000", 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.638e-06 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062793+0000", 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019559246999999998 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062810+0000", 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7431e-05 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062819+0000", 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.329e-06 2024-09-14T22:53:32.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062890+0000", 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0605999999999995e-05 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063027+0000", 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000136606 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063504+0000", 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000477875 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086724+0000", 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023220082 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087212+0000", 2024-09-14T22:53:32.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048734800000000002 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087262+0000", 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0652999999999998e-05 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087311+0000", 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8560999999999998e-05 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.648 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:64130 3.7 e38/24 3:e0c199cd:::rbd_data.1135c0c4962c.0000000000000001:head v 38'45, mlcod=38'43)", 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.4964420199999999, 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.040529201000000001, 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045691+0000", 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054194+0000", 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085033850000000005 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054219+0000", 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4533999999999999e-05 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054231+0000", 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2617e-05 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054328+0000", 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6978000000000006e-05 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054355+0000", 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6302e-05 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086122+0000", 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.031767008999999999 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086219+0000", 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7361999999999994e-05 2024-09-14T22:53:32.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:76917 3.15 3:ab9de9be:::rbd_header.113535cec067:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.664266885, 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047974171000000003, 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:32.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 76917 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:32.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877866+0000", 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7899999999999998e-07 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877869+0000", 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4429999999999999e-06 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877875+0000", 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5160000000000002e-06 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877879+0000", 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.206e-06 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877906+0000", 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7254999999999999e-05 2024-09-14T22:53:32.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877949+0000", 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2830000000000002e-05 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.925839+0000", 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047890342000000002 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:32.659 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:32.659 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:32.660 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:32.661 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:32.659+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:32.761 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:32.759+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:32.790 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:32.790 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:32.791 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:32.791 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:32.802 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:32.862 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:32.860+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:32.870 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:53:32.913 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:32.913 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:32.913 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:32.913 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:32.914 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:32.923 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:32.962 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:32.961+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:33.017 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:33.031 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1795995640000001, 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:53:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1786619740000006, 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:53:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:53:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1785706109999996, 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:53:33.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:33.048 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1779257259999998, 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.049 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:53:33.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1774383470000007, 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.054 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.055 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:53:33.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.058 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1765510720000005, 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:53:33.059 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.060 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:53:33.061 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.062 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.063 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1759586479999999, 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.066 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:53:33.067 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:53:33.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:53:33.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.070 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1752258320000006, 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.071 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.072 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.073 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:53:33.074 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.075 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1751266830000002, 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.076 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.077 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.078 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:53:33.079 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.080 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1748082249999996, 2024-09-14T22:53:33.081 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.082 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:53:33.083 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:53:33.084 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1742987310000004, 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.087 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.089 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.091 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1742408720000004, 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.092 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:33.093 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.094 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.095 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:53:33.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.097 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1734482489999998, 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.098 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:53:33.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:53:33.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.102 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1727728099999997, 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.103 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.105 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:53:33.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.107 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1726395000000007, 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:53:33.108 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.109 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:33.110 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:53:33.111 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.113 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1720225769999999, 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.114 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:53:33.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:53:33.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.118 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1719557199999997, 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.119 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.122 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:53:33.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:33.124 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1715049089999994, 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:53:33.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1713846760000006, 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:53:33.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.134 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.1700665489999995, 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:33.135 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:33.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:33.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:33.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:53:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:53:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:33.140 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:33.140 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:33.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:33.142 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:33.142 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:33.143 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:33.140+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:33.155 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:33.155 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:33.171 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:33.188 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:33.188 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:33.205 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:33.205 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:33.222 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:33.222 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:33.239 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:33.239 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:33.242 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:33.241+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:33.255 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:33.260 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:33.260 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:33.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:33.353 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:33.352+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:33.412 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:33.412 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:33.420 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:33.461 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:33.460+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:33.544 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:33.543+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:33.563 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:33.563 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:33.571 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:33.645 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:33.644+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:33.674 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:33.674 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:33.683 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:33.746 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:33.745+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:33.747 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:53:33.790 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:33.790 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:33.799 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:33.883 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:33.882+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:33.942 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:33.943 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:33.948 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:33.947+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:33.950 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:34.049 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:34.047+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:34.108 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:34.108 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:34.118 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:34.159 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:34.158+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:34.174 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T22:53:34.215 INFO:teuthology.orchestra.run.smithi076.stdout:789 read 0xb35e3cf thru 0xb3686ef (0xa321 bytes) 2024-09-14T22:53:34.218 INFO:teuthology.orchestra.run.smithi076.stdout:790 punch from 0x7c0f09f to 0x7c12e27, (0x3d88 bytes) 2024-09-14T22:53:34.222 INFO:teuthology.orchestra.run.smithi076.stdout:795 writesame 0xa48637a thru 0xa493291 data_size 0x8d(0xcf18 bytes) 2024-09-14T22:53:34.250 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:34.249+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:34.285 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:34.285 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:34.295 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:34.351 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:34.350+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:34.442 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:34.442 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:34.451 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:34.452 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:34.451+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:34.553 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:34.552+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:34.554 INFO:teuthology.orchestra.run.smithi076.stdout:796 punch from 0x15fdf80 to 0x160aade, (0xcb5e bytes) 2024-09-14T22:53:34.554 INFO:teuthology.orchestra.run.smithi076.stdout:797 read 0x4949574 thru 0x494fe73 (0x6900 bytes) 2024-09-14T22:53:34.554 INFO:teuthology.orchestra.run.smithi076.stdout:798 write 0x69485e3 thru 0x69550e8 (0xcb06 bytes) 2024-09-14T22:53:34.563 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:34.563 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:34.572 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:34.653 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:34.652+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:34.718 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:34.718 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:34.727 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:34.770 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:34.769+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:34.837 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:34.837 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:34.846 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:34.856 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:34.854+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:34.957 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:34.955+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:34.990 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:34.990 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:34.999 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:35.058 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:35.057+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:35.141 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:35.141 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:35.150 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:35.159 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:35.158+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:35.253 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:35.253 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:35.260 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:35.259+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:35.261 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:35.268 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:35.360 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:35.359+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:35.363 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:35.372 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:35.388 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:35.388 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:35.404 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:35.404 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:35.420 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:35.420 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:35.436 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:35.436 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:35.452 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:35.453 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:35.461 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:35.460+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:35.469 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:35.469 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:35.485 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:35.485 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:35.502 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:35.503 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:35.518 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:35.518 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:35.518 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:35.527 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:35.568 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:35.567+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:35.663 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:35.662+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:35.671 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:35.671 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.587854098999999, 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:53:35.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.676 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.587636203000001, 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:53:35.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.584952556999999, 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:53:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.584794142, 2024-09-14T22:53:35.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:35.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:53:35.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.584766876, 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.584559734999999, 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:53:35.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.584525470999999, 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:35.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.703 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.582893425, 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:53:35.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:53:35.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.582868812999999, 2024-09-14T22:53:35.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:53:35.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:53:35.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.582734527, 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:53:35.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:53:35.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.581684818999999, 2024-09-14T22:53:35.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:53:35.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.581582634, 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.722 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:53:35.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.581558729999999, 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.726 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.730 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.581525210000001, 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:53:35.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.581370564, 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.581329390000001, 2024-09-14T22:53:35.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:53:35.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:53:35.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.743 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.581301229999999, 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.744 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:53:35.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:53:35.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:53:35.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.581111839, 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.748 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:35.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:35.752 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 11.665022012, 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:35.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:53:35.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:53:35.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 11.665002181, 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.757 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:35.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:35.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:35.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:53:35.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:53:35.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.761 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:35.761 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:35.761 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:35.761 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:53:35.762 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:35.763 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:35.763+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:35.864 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:35.863+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:35.867 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:35.867 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:35.867 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:35.867 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:35.877 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:35.963 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:35.963 INFO:teuthology.orchestra.run.smithi076.stdout:802 punch from 0xc248c35 to 0xc2524f4, (0x98bf bytes) 2024-09-14T22:53:35.965 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:35.964+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:35.965 INFO:teuthology.orchestra.run.smithi076.stdout:804 trunc from 0xc3ba046 to 0xe1f8e66 2024-09-14T22:53:35.971 INFO:teuthology.orchestra.run.smithi076.stdout:805 read 0x7df94b4 thru 0x7dfc6c9 (0x3216 bytes) 2024-09-14T22:53:35.972 INFO:teuthology.orchestra.run.smithi076.stdout:807 write 0xe1af072 thru 0xe1b88b7 (0x9846 bytes) 2024-09-14T22:53:35.975 INFO:teuthology.orchestra.run.smithi076.stdout:811 read 0x5614adc thru 0x5616538 (0x1a5d bytes) 2024-09-14T22:53:35.975 INFO:teuthology.orchestra.run.smithi076.stdout:812 writesame 0x83c3bb6 thru 0x83d03a1 data_size 0x1e(0xc7ec bytes) 2024-09-14T22:53:35.988 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:35.988 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:35.988 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:35.988 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:35.988 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:35.999 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:36.066 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:36.065+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:36.166 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:36.165+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.676895459999997, 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.187 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:53:36.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:53:36.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5811 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.799658574999999, 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035921055, 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.194 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5811 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385013+0000", 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5199999999999998e-07 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385023+0000", 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0035e-05 2024-09-14T22:53:36.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385029+0000", 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5250000000000002e-06 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.385035+0000", 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2460000000000003e-06 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420764+0000", 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035729819000000003 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420840+0000", 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5784000000000005e-05 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.420934+0000", 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3294000000000005e-05 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.198 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5864 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.797962606999999, 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035943627999999998, 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.199 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5864 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386708+0000", 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386920+0000", 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021137200000000001 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386923+0000", 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4230000000000001e-06 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386932+0000", 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4290000000000001e-06 2024-09-14T22:53:36.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386938+0000", 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0839999999999997e-06 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422480+0000", 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035542705000000001 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422559+0000", 2024-09-14T22:53:36.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8385000000000003e-05 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422652+0000", 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.323e-05 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:5865 3.d 3:b14a1d11:::rbd_data.11354ec77855.000000000000000d:6 [read 1046528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:36.203 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.797724119000002, 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035838531, 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 5865 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386947+0000", 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0399999999999996e-07 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386950+0000", 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2749999999999998e-06 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386956+0000", 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2090000000000002e-06 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.386961+0000", 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1959999999999999e-06 2024-09-14T22:53:36.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422683+0000", 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035721387 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422727+0000", 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.401e-05 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:17.422785+0000", 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8950000000000003e-05 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53826 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 913408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.766037003999999, 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035814618999999999, 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.208 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53826 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418634+0000", 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418635+0000", 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4300000000000003e-07 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418637+0000", 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0090000000000002e-06 2024-09-14T22:53:36.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418643+0000", 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1109999999999996e-06 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418648+0000", 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.152e-06 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454259+0000", 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035610615999999998 2024-09-14T22:53:36.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454326+0000", 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7565999999999997e-05 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454449+0000", 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012262199999999999 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53830 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.766017301, 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036000748999999999, 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53830 2024-09-14T22:53:36.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418654+0000", 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999996e-07 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418656+0000", 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0729999999999999e-06 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418663+0000", 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418668+0000", 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1100000000000002e-06 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454478+0000", 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035809797999999997 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454544+0000", 2024-09-14T22:53:36.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6089999999999999e-05 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454655+0000", 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110788 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53831 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:36.217 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.765996812000001, 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036273573000000003, 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53831 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418674+0000", 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418675+0000", 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418677+0000", 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.086e-06 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418684+0000", 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.754e-06 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418689+0000", 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6300000000000003e-06 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454701+0000", 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036011950000000001 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454806+0000", 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104558 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454948+0000", 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142177 2024-09-14T22:53:36.221 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53833 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 915456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.765973986000001, 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.03648067, 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.222 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53833 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418697+0000", 2024-09-14T22:53:36.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418698+0000", 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418700+0000", 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.091e-06 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418706+0000", 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6850000000000004e-06 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418712+0000", 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2059999999999998e-06 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.454991+0000", 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036279441000000003 2024-09-14T22:53:36.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455065+0000", 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3617999999999999e-05 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455178+0000", 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000113205 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:53860 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 862208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.765952931999999, 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036670083999999999, 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 53860 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418718+0000", 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418719+0000", 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6899999999999997e-07 2024-09-14T22:53:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418725+0000", 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1569999999999997e-06 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418733+0000", 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9009999999999994e-06 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.418738+0000", 2024-09-14T22:53:36.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.553e-06 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455206+0000", 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036467558999999997 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455273+0000", 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7378000000000004e-05 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.455388+0000", 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000114967 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54365 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 737280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.752940321000001, 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035967086000000002, 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.231 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54365 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431731+0000", 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0600000000000001e-07 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431733+0000", 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9230000000000001e-06 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431742+0000", 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5669999999999995e-06 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431747+0000", 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.383e-06 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467514+0000", 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035766569999999998 2024-09-14T22:53:36.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467582+0000", 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8373000000000004e-05 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467698+0000", 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000115864 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54366 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.752917134, 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036163224000000001, 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54366 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.236 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431754+0000", 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6199999999999998e-07 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431757+0000", 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.272e-06 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431764+0000", 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8240000000000002e-06 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431769+0000", 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1410000000000004e-06 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467728+0000", 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035959526999999998 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467797+0000", 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8681000000000002e-05 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467917+0000", 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012031699999999999 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54367 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.752896171, 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036385946000000002, 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.240 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54367 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431775+0000", 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0499999999999999e-07 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431777+0000", 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9290000000000001e-06 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431784+0000", 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3790000000000003e-06 2024-09-14T22:53:36.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431789+0000", 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1660000000000002e-06 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.467947+0000", 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036158542000000002 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468020+0000", 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2429999999999996e-05 2024-09-14T22:53:36.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468161+0000", 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014109499999999999 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54368 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.752874728, 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036591052999999998, 2024-09-14T22:53:36.244 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54368 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431796+0000", 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431797+0000", 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431803+0000", 2024-09-14T22:53:36.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.438e-06 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431812+0000", 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5520000000000005e-06 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431817+0000", 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5389999999999998e-06 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468191+0000", 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036374191 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468265+0000", 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3385000000000003e-05 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468387+0000", 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012255400000000001 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54369 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.752846377000001, 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036784182999999998, 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54369 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.249 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431825+0000", 2024-09-14T22:53:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4200000000000001e-07 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431827+0000", 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.221e-06 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431834+0000", 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5400000000000001e-06 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431839+0000", 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7570000000000004e-06 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468417+0000", 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036577927000000003 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468489+0000", 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2412999999999999e-05 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468609+0000", 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119883 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54370 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 765952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.752826553, 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036997090000000003, 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.253 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54370 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431845+0000", 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2100000000000002e-07 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431847+0000", 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9750000000000001e-06 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431853+0000", 2024-09-14T22:53:36.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.173e-06 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431858+0000", 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8810000000000004e-06 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468641+0000", 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036783320000000001 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468718+0000", 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6417999999999999e-05 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468842+0000", 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012390200000000001 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54371 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.257 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.752807237000001, 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037201153000000001, 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54371 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:36.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431864+0000", 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2300000000000002e-07 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431866+0000", 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0700000000000001e-06 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431875+0000", 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.225e-06 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431880+0000", 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1189999999999996e-06 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468871+0000", 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.036991375999999999 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.468948+0000", 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6991999999999997e-05 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469065+0000", 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011694800000000001 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.261 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:54373 3.11 3:8ecd1a3d:::rbd_data.11354ec77855.0000000000000011:6 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.752784908000001, 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037377190999999997, 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 54373 2024-09-14T22:53:36.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431886+0000", 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431887+0000", 2024-09-14T22:53:36.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431889+0000", 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0470000000000001e-06 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431895+0000", 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4760000000000003e-06 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.431900+0000", 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8539999999999996e-06 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469091+0000", 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.037190894000000002 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469151+0000", 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9867999999999997e-05 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:23.469263+0000", 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011267700000000001 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64133 3.f 3:f670552a:::rbd_data.1135c0c4962c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 3276800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.141439438999999, 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.044079376000000003, 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.266 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64133 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043232+0000", 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.043233+0000", 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.638e-06 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062793+0000", 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019559246999999998 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062810+0000", 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7431e-05 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062819+0000", 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.329e-06 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.062890+0000", 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0605999999999995e-05 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063027+0000", 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000136606 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.063504+0000", 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000477875 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086724+0000", 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023220082 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087212+0000", 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048734800000000002 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087262+0000", 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0652999999999998e-05 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.087311+0000", 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8560999999999998e-05 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.271 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:64130 3.7 e38/24 3:e0c199cd:::rbd_data.1135c0c4962c.0000000000000001:head v 38'45, mlcod=38'43)", 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.138981153, 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.040529201000000001, 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045690+0000", 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.045691+0000", 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0139999999999999e-06 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054194+0000", 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085033850000000005 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054219+0000", 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4533999999999999e-05 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054231+0000", 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2617e-05 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054328+0000", 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6978000000000006e-05 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.054355+0000", 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6302e-05 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086122+0000", 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.031767008999999999 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.086219+0000", 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7361999999999994e-05 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:76917 3.15 3:ab9de9be:::rbd_header.113535cec067:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3068060179999996, 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047974171000000003, 2024-09-14T22:53:36.276 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 76917 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877865+0000", 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877866+0000", 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7899999999999998e-07 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877869+0000", 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4429999999999999e-06 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877875+0000", 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5160000000000002e-06 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877879+0000", 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.206e-06 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877906+0000", 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7254999999999999e-05 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.877949+0000", 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2830000000000002e-05 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:26.925839+0000", 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.047890342000000002 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:36.280 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:36.282 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:36.282 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:36.280+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:36.311 INFO:teuthology.orchestra.run.smithi076.stdout:813 write 0x2e2ae06 thru 0x2e2c540 (0x173b bytes) 2024-09-14T22:53:36.383 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:36.381+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:36.418 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:36.418 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:36.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:36.513 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:36.512+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:36.571 INFO:teuthology.orchestra.run.smithi076.stdout:814 read 0x810bb23 thru 0x811bb15 (0xfff3 bytes) 2024-09-14T22:53:36.571 INFO:teuthology.orchestra.run.smithi076.stdout:815 punch from 0x17e096a to 0x17e7883, (0x6f19 bytes) 2024-09-14T22:53:36.571 INFO:teuthology.orchestra.run.smithi076.stdout:816 punch from 0xa18c89b to 0xa198475, (0xbbda bytes) 2024-09-14T22:53:36.571 INFO:teuthology.orchestra.run.smithi076.stdout:817 writesame 0x188b048 thru 0x188bad5 data_size 0x7(0xa8e bytes) 2024-09-14T22:53:36.572 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:36.572 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:36.572 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:36.572 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:36.572 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:36.585 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:36.586 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:36.583+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:36.685 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:36.684+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.923852773, 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:53:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:53:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.922915183000001, 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:53:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.92282382, 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:53:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.922178935, 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:53:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.921691556000001, 2024-09-14T22:53:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:53:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:53:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.920804281000001, 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.920211857, 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:53:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:53:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.919479041000001, 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:53:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:53:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.919379892, 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.919061434, 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:53:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:53:36.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:53:36.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.91855194, 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.822 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:53:36.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.826 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.918494081, 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:53:36.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:53:36.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.917701458, 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.831 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:53:36.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:53:36.835 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.917026019, 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.836 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.916892709000001, 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.840 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:53:36.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:53:36.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.916275786, 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.848 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.916208929, 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.849 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.915758117999999, 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.853 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:53:36.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:53:36.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.915637885000001, 2024-09-14T22:53:36.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.914319758, 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.865 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.866 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.866 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.866 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.866 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:36.869 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:36.870 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:36.867+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:36.870 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:53:36.871 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:36.871 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:36.871 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T22:53:36.871 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.572138 2024-09-14T22:53:36.872 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.572138149238567 2024-09-14T22:53:36.969 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:36.968+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:36.985 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:121889 3.19 3:9bb33de6:::rbd_data.1135d503d0a.0000000000000000:head [call rbd.sparse_copyup in=16774570b,set-alloc-hint object_size 16777216 write_size 16777216,write 13152328~2702 in=2702b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.925451+0000", 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.059381380999999997, 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.059418606999999998, 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "waiting for sub ops", 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 121889 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.925451+0000", 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.925451+0000", 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.925453+0000", 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.962e-06 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.933470+0000", 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0080175799999999998 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.933499+0000", 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8042999999999999e-05 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.933508+0000", 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5719999999999992e-06 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.933594+0000", 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5736999999999996e-05 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.933758+0000", 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000164594 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 3", 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.935181+0000", 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0014221819999999999 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 1 2024-09-14T22:53:36.990 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:36.991 INFO:teuthology.orchestra.run.smithi076.stdout:818 punch from 0xbc36491 to 0xbc441e4, (0xdd53 bytes) 2024-09-14T22:53:36.991 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:36.991 INFO:teuthology.orchestra.run.smithi076.stdout:820 write 0x6313b7c thru 0x631fbff (0xc084 bytes) 2024-09-14T22:53:36.991 INFO:teuthology.orchestra.run.smithi076.stdout:821 writesame 0x3ac2e1 thru 0x3adbf7 data_size 0x1(0x1917 bytes) 2024-09-14T22:53:36.999 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:37.070 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:37.068+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:37.144 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:37.145 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:37.145 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:37.145 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:37.145 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:37.158 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:37.203 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:37.202+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:37.272 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:37.270+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:37.292 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.292933528, 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29291449000000003, 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:53:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29287853400000002, 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:53:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29285741399999998, 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:53:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.29165049999999998, 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:53:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:53:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:53:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.289821053, 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:53:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.28926676600000001, 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:53:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:53:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:53:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.28728204800000001, 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:53:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:53:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:53:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.286929728, 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:53:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.286646285, 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:53:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:53:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.28635818800000001, 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:53:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.28598336000000002, 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:53:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:53:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.28566254499999999, 2024-09-14T22:53:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.27197731000000003, 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:53:37.354 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.27164250099999998, 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.355 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:37.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:53:37.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:53:37.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:37.359 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.27131233900000001, 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:37.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:53:37.363 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.27109155299999999, 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.364 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.27082225199999999, 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:53:37.368 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.372 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.27063515599999999, 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:53:37.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:53:37.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.270316997, 2024-09-14T22:53:37.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:53:37.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:53:37.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:37.380 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:37.382 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:37.382 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:37.380+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:37.431 INFO:teuthology.orchestra.run.smithi076.stdout:823 punch from 0x4747c0e to 0x475219a, (0xa58c bytes) 2024-09-14T22:53:37.432 INFO:teuthology.orchestra.run.smithi076.stdout:824 writesame 0x372cebd thru 0x3737e8b data_size 0x1(0xafcf bytes) 2024-09-14T22:53:37.482 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:37.481+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:37.514 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:37.569 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:37.569 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:37.569 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:37.569 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:37.578 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:37.582 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:37.581+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:37.655 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:37.655 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:37.673 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:37.673 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:37.683 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:37.682+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:37.687 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:37.687 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:37.687 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:37.687 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:37.687 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:37.689 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:37.690 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:37.696 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:37.706 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:37.706 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:37.721 INFO:teuthology.orchestra.run.smithi076.stdout:825 writesame 0x570288a thru 0x570ed29 data_size 0x58(0xc4a0 bytes) 2024-09-14T22:53:37.724 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:37.724 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:37.746 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:37.746 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:37.764 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:37.764 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:37.783 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:37.783 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:37.784 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:37.782+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:37.820 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:37.820 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:37.830 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:37.870 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.0 primary-affinity to 0.572138 (8374952) 2024-09-14T22:53:37.884 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:37.883+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:37.977 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:37.977 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:37.985 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:37.986 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:37.984+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:38.085 INFO:teuthology.orchestra.run.smithi076.stdout:826 read 0x4c79759 thru 0x4c7b98e (0x2236 bytes) 2024-09-14T22:53:38.085 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:38.085 INFO:teuthology.orchestra.run.smithi076.stdout:828 write 0x8729147 thru 0x872b33a (0x21f4 bytes) 2024-09-14T22:53:38.085 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:38.084+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:38.086 INFO:teuthology.orchestra.run.smithi076.stdout:829 write 0xb44465 thru 0xb4a029 (0x5bc5 bytes) 2024-09-14T22:53:38.088 INFO:teuthology.orchestra.run.smithi076.stdout:830 read 0xb674e34 thru 0xb67cad1 (0x7c9e bytes) 2024-09-14T22:53:38.092 INFO:teuthology.orchestra.run.smithi076.stdout:832 punch from 0x52a614a to 0x52b4fc4, (0xee7a bytes) 2024-09-14T22:53:38.094 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:38.094 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:38.095 INFO:teuthology.orchestra.run.smithi076.stdout:833 writesame 0xe705648 thru 0xe70d3b0 data_size 0x1(0x7d69 bytes) 2024-09-14T22:53:38.096 INFO:teuthology.orchestra.run.smithi076.stdout:834 read 0x711aa67 thru 0x71243ff (0x9999 bytes) 2024-09-14T22:53:38.097 INFO:teuthology.orchestra.run.smithi076.stdout:835 punch from 0x34c7367 to 0x34c854e, (0x11e7 bytes) 2024-09-14T22:53:38.097 INFO:teuthology.orchestra.run.smithi076.stdout:836 read 0x613d2c9 thru 0x613fcc8 (0x2a00 bytes) 2024-09-14T22:53:38.106 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:38.112 INFO:teuthology.orchestra.run.smithi076.stdout:838 trunc from 0xe70d3b1 to 0x202a785 2024-09-14T22:53:38.133 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:38.133 INFO:teuthology.orchestra.run.smithi076.stdout:841 punch from 0xbbb3da to 0xbc9555, (0xe17b bytes) 2024-09-14T22:53:38.133 INFO:teuthology.orchestra.run.smithi076.stdout:843 punch from 0x144411d to 0x144e415, (0xa2f8 bytes) 2024-09-14T22:53:38.135 INFO:teuthology.orchestra.run.smithi076.stdout:845 trunc from 0x202a785 to 0x4196fb3 2024-09-14T22:53:38.148 INFO:teuthology.orchestra.run.smithi076.stdout:846 write 0x5e02f12 thru 0x5e06c5e (0x3d4d bytes) 2024-09-14T22:53:38.186 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:38.185+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:38.212 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:38.212 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:38.221 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:38.287 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:38.285+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:38.324 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:38.324 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:38.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:38.388 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:38.385+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:38.436 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:38.436 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:38.445 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:38.488 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:38.487+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:38.547 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:38.547 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:38.556 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:38.589 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:38.587+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:38.690 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:38.689+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:38.698 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:38.698 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:38.706 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:38.791 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:38.789+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:38.810 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:38.810 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:38.819 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:38.883 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:53:38.897 INFO:teuthology.orchestra.run.smithi076.stdout:847 writesame 0x78e4e9a thru 0x78e96a6 data_size 0x23(0x480d bytes) 2024-09-14T22:53:38.897 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:38.897 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:38.895+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:38.902 INFO:teuthology.orchestra.run.smithi076.stdout:849 trunc from 0x78e96a7 to 0x4b7bf3c 2024-09-14T22:53:38.922 INFO:teuthology.orchestra.run.smithi076.stdout:850 write 0x7ed1c51 thru 0x7ed7793 (0x5b43 bytes) 2024-09-14T22:53:38.926 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:38.926 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:38.935 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:38.939 INFO:teuthology.orchestra.run.smithi076.stdout:851 write 0xd8019df thru 0xd80dec5 (0xc4e7 bytes) 2024-09-14T22:53:38.944 INFO:teuthology.orchestra.run.smithi076.stdout:852 read 0x6f01f78 thru 0x6f062b2 (0x433b bytes) 2024-09-14T22:53:38.944 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:38.944 INFO:teuthology.orchestra.run.smithi076.stdout:854 write 0xc4afc5c thru 0xc4b1f04 (0x22a9 bytes) 2024-09-14T22:53:38.946 INFO:teuthology.orchestra.run.smithi076.stdout:856 punch from 0x464b186 to 0x464c58c, (0x1406 bytes) 2024-09-14T22:53:38.946 INFO:teuthology.orchestra.run.smithi076.stdout:858 punch from 0xc433d20 to 0xc4399fa, (0x5cda bytes) 2024-09-14T22:53:38.946 INFO:teuthology.orchestra.run.smithi076.stdout:859 read 0x36afa2e thru 0x36b0b11 (0x10e4 bytes) 2024-09-14T22:53:38.946 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:38.946 INFO:teuthology.orchestra.run.smithi076.stdout:862 writesame 0x3be9e12 thru 0x3bf9025 data_size 0x2(0xf214 bytes) 2024-09-14T22:53:38.976 INFO:teuthology.orchestra.run.smithi076.stdout:863 trunc from 0xd80dec6 to 0x38bafca 2024-09-14T22:53:38.993 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:38.991+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:38.998 INFO:teuthology.orchestra.run.smithi076.stdout:864 writesame 0x60e9157 thru 0x60ee58b data_size 0x1(0x5435 bytes) 2024-09-14T22:53:38.999 INFO:teuthology.orchestra.run.smithi076.stdout:866 read 0x5ec20f8 thru 0x5ec3185 (0x108e bytes) 2024-09-14T22:53:38.999 INFO:teuthology.orchestra.run.smithi076.stdout:867 punch from 0x74adb8 to 0x7501ab, (0x53f3 bytes) 2024-09-14T22:53:38.999 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:38.999 INFO:teuthology.orchestra.run.smithi076.stdout:869 write 0x288cc80 thru 0x2894281 (0x7602 bytes) 2024-09-14T22:53:39.002 INFO:teuthology.orchestra.run.smithi076.stdout:870 trunc from 0x60ee58c to 0x8300111 2024-09-14T22:53:39.008 INFO:teuthology.orchestra.run.smithi076.stdout:871 writesame 0x523c2ba thru 0x524c18a data_size 0x1(0xfed1 bytes) 2024-09-14T22:53:39.041 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:39.041 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:39.045 INFO:teuthology.orchestra.run.smithi076.stdout:872 read 0x5fcf788 thru 0x5fd4dcd (0x5646 bytes) 2024-09-14T22:53:39.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:39.055 INFO:teuthology.orchestra.run.smithi076.stdout:873 writesame 0xacc70c0 thru 0xacd00bf data_size 0x6(0x9000 bytes) 2024-09-14T22:53:39.058 INFO:teuthology.orchestra.run.smithi076.stdout:874 trunc from 0xacd00c0 to 0xd746db4 2024-09-14T22:53:39.064 INFO:teuthology.orchestra.run.smithi076.stdout:875 punch from 0xd5c1d57 to 0xd5d126c, (0xf515 bytes) 2024-09-14T22:53:39.064 INFO:teuthology.orchestra.run.smithi076.stdout:877 read 0x34017db thru 0x3403dc1 (0x25e7 bytes) 2024-09-14T22:53:39.065 INFO:teuthology.orchestra.run.smithi076.stdout:878 punch from 0x6532b23 to 0x65380ee, (0x55cb bytes) 2024-09-14T22:53:39.065 INFO:teuthology.orchestra.run.smithi076.stdout:879 punch from 0xc114946 to 0xc114f6a, (0x624 bytes) 2024-09-14T22:53:39.076 INFO:teuthology.orchestra.run.smithi076.stdout:880 trunc from 0xd746db4 to 0x416b4de 2024-09-14T22:53:39.085 INFO:teuthology.orchestra.run.smithi076.stdout:truncating to largest ever: 0xec1a452 2024-09-14T22:53:39.094 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:39.093+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:39.098 INFO:teuthology.orchestra.run.smithi076.stdout:881 trunc from 0x416b4de to 0xec1a452 2024-09-14T22:53:39.106 INFO:teuthology.orchestra.run.smithi076.stdout:882 trunc from 0xec1a452 to 0xd278bfc 2024-09-14T22:53:39.111 INFO:teuthology.orchestra.run.smithi076.stdout:883 punch from 0x937ffc3 to 0x9380352, (0x38f bytes) 2024-09-14T22:53:39.111 INFO:teuthology.orchestra.run.smithi076.stdout:884 writesame 0x9ab8938 thru 0x9ac4cd9 data_size 0x22(0xc3a2 bytes) 2024-09-14T22:53:39.115 INFO:teuthology.orchestra.run.smithi076.stdout:885 read 0x8bf1329 thru 0x8c002af (0xef87 bytes) 2024-09-14T22:53:39.115 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:39.115 INFO:teuthology.orchestra.run.smithi076.stdout:887 read 0xc521ca9 thru 0xc528eba (0x7212 bytes) 2024-09-14T22:53:39.116 INFO:teuthology.orchestra.run.smithi076.stdout:888 read 0xae12f33 thru 0xae1db93 (0xac61 bytes) 2024-09-14T22:53:39.116 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:39.116 INFO:teuthology.orchestra.run.smithi076.stdout:890 read 0x1a73faf thru 0x1a7d9ad (0x99ff bytes) 2024-09-14T22:53:39.116 INFO:teuthology.orchestra.run.smithi076.stdout:894 punch from 0x672b4ba to 0x6738a47, (0xd58d bytes) 2024-09-14T22:53:39.116 INFO:teuthology.orchestra.run.smithi076.stdout:895 read 0x5864495 thru 0x5867537 (0x30a3 bytes) 2024-09-14T22:53:39.116 INFO:teuthology.orchestra.run.smithi076.stdout:899 read 0x1562038 thru 0x156da4f (0xba18 bytes) 2024-09-14T22:53:39.117 INFO:teuthology.orchestra.run.smithi076.stdout:900 read 0x17fb292 thru 0x17fe2d5 (0x3044 bytes) 2024-09-14T22:53:39.118 INFO:teuthology.orchestra.run.smithi076.stdout:901 write 0x7d340d3 thru 0x7d40f9d (0xcecb bytes) 2024-09-14T22:53:39.124 INFO:teuthology.orchestra.run.smithi076.stdout:902 write 0xa6372d3 thru 0xa645c5a (0xe988 bytes) 2024-09-14T22:53:39.130 INFO:teuthology.orchestra.run.smithi076.stdout:903 write 0xaab7a6 thru 0xaaf761 (0x3fbc bytes) 2024-09-14T22:53:39.139 INFO:teuthology.orchestra.run.smithi076.stdout:904 trunc from 0xd278bfc to 0x6db472d 2024-09-14T22:53:39.157 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:39.157 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:39.159 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:39.159 INFO:teuthology.orchestra.run.smithi076.stdout:906 writesame 0x5eb0b66 thru 0x5eb632f data_size 0x11(0x57ca bytes) 2024-09-14T22:53:39.161 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:39.161 INFO:teuthology.orchestra.run.smithi076.stdout:909 read 0xd68c41 thru 0xd6d056 (0x4416 bytes) 2024-09-14T22:53:39.162 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:39.162 INFO:teuthology.orchestra.run.smithi076.stdout:912 write 0x5235968 thru 0x523e212 (0x88ab bytes) 2024-09-14T22:53:39.166 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:39.166 INFO:teuthology.orchestra.run.smithi076.stdout:913 trunc from 0x6db472d to 0x984094c 2024-09-14T22:53:39.172 INFO:teuthology.orchestra.run.smithi076.stdout:914 punch from 0x41a0780 to 0x41a9464, (0x8ce4 bytes) 2024-09-14T22:53:39.172 INFO:teuthology.orchestra.run.smithi076.stdout:915 punch from 0x95e8101 to 0x95f3c31, (0xbb30 bytes) 2024-09-14T22:53:39.172 INFO:teuthology.orchestra.run.smithi076.stdout:916 read 0x7917937 thru 0x791c392 (0x4a5c bytes) 2024-09-14T22:53:39.173 INFO:teuthology.orchestra.run.smithi076.stdout:917 punch from 0x8fb136c to 0x8fb8bde, (0x7872 bytes) 2024-09-14T22:53:39.173 INFO:teuthology.orchestra.run.smithi076.stdout:918 clone 12 order 23 su 262144 sc 5 2024-09-14T22:53:39.207 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:39.206+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:39.295 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:39.294+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:39.309 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:39.309 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:39.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:39.396 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:39.394+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:39.462 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:39.462 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:39.471 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:39.515 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:39.514+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:39.574 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:39.574 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:39.583 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:39.624 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:39.623+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:39.699 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:39.698+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:39.726 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:39.726 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:39.735 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:39.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:39.799 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:39.799+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:39.849 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:39.849 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:39.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:39.885 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:53:39.901 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:39.899+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:39.928 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:39.928 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:39.946 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:39.947 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:39.964 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:39.964 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:39.976 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:39.976 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:39.977 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:39.977 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:39.977 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:39.981 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:39.982 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:39.986 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:39.999 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:39.999 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:40.000 INFO:teuthology.orchestra.run.smithi076.stdout:leaving image image_client.0-clone11 intact 2024-09-14T22:53:40.002 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:40.000+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:40.018 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:40.018 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:40.036 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:40.036 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:40.053 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:40.053 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:40.106 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:40.104+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:40.112 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.260367808000002, 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:53:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:53:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.259430217999999, 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:53:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.259338854999999, 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:53:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.258693969999999, 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:53:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.258206591, 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.257319316, 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:53:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:53:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.256726892, 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.255994076, 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:53:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:53:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.255894927, 2024-09-14T22:53:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:53:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:53:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:53:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.255576469000001, 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:53:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.255066974999998, 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.255009116, 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.166 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:53:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.254216493000001, 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.253541053999999, 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:53:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.253407744, 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.252790821000001, 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:53:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:53:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:53:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:53:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.252723964000001, 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.194 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.252273153000001, 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:53:40.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:53:40.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:53:40.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.25215292, 2024-09-14T22:53:40.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:53:40.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.250834792999999, 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:53:40.205 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:53:40.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:53:40.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.210 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:40.212 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:40.213 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T22:53:40.213 INFO:teuthology.orchestra.run.smithi076.stdout:919 read 0x746c5c1 thru 0x747a102 (0xdb42 bytes) 2024-09-14T22:53:40.213 INFO:teuthology.orchestra.run.smithi076.stdout:920 write 0xe839b1d thru 0xe8444ae (0xa992 bytes) 2024-09-14T22:53:40.213 INFO:teuthology.orchestra.run.smithi076.stdout:921 read 0x53a8e5c thru 0x53a9083 (0x228 bytes) 2024-09-14T22:53:40.213 INFO:teuthology.orchestra.run.smithi076.stdout:922 read 0xd5f421 thru 0xd6cbbd (0xd79d bytes) 2024-09-14T22:53:40.214 INFO:teuthology.orchestra.run.smithi076.stdout:923 punch from 0xdc29eeb to 0xdc35365, (0xb47a bytes) 2024-09-14T22:53:40.214 INFO:teuthology.orchestra.run.smithi076.stdout:924 write 0x78ff81d thru 0x790ab93 (0xb377 bytes) 2024-09-14T22:53:40.214 INFO:teuthology.orchestra.run.smithi076.stdout:926 writesame 0x7d1b912 thru 0x7d1c237 data_size 0x1(0x926 bytes) 2024-09-14T22:53:40.214 INFO:teuthology.orchestra.run.smithi076.stdout:927 write 0xc1c837 thru 0xc22adb (0x62a5 bytes) 2024-09-14T22:53:40.214 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:40.209+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:40.237 INFO:teuthology.orchestra.run.smithi076.stdout:928 write 0x11cfdfa thru 0x11debdd (0xede4 bytes) 2024-09-14T22:53:40.271 INFO:teuthology.orchestra.run.smithi076.stdout:930 punch from 0x9439164 to 0x943ccba, (0x3b56 bytes) 2024-09-14T22:53:40.271 INFO:teuthology.orchestra.run.smithi076.stdout:931 punch from 0xdad9f21 to 0xdae30a2, (0x9181 bytes) 2024-09-14T22:53:40.272 INFO:teuthology.orchestra.run.smithi076.stdout:932 punch from 0x29cc15a to 0x29ce6d8, (0x257e bytes) 2024-09-14T22:53:40.289 INFO:teuthology.orchestra.run.smithi076.stdout:933 trunc from 0xe8444af to 0xe1db72 2024-09-14T22:53:40.313 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:40.311+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:40.348 INFO:teuthology.orchestra.run.smithi076.stdout:935 read 0xaeb0c4 thru 0xaf50be (0x9ffb bytes) 2024-09-14T22:53:40.348 INFO:teuthology.orchestra.run.smithi076.stdout:936 read 0xadf8f1 thru 0xae28df (0x2fef bytes) 2024-09-14T22:53:40.365 INFO:teuthology.orchestra.run.smithi076.stdout:937 write 0x8dc52e2 thru 0x8dd3366 (0xe085 bytes) 2024-09-14T22:53:40.367 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.373 INFO:teuthology.orchestra.run.smithi076.stdout:939 trunc from 0x8dd3367 to 0xceeae31 2024-09-14T22:53:40.373 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:40.373 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:40.373 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:40.373 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:40.377 INFO:teuthology.orchestra.run.smithi076.stdout:942 read 0x1f2941b thru 0x1f2fd37 (0x691d bytes) 2024-09-14T22:53:40.377 INFO:teuthology.orchestra.run.smithi076.stdout:943 writesame 0x6b11ee8 thru 0x6b1c2c1 data_size 0x3(0xa3da bytes) 2024-09-14T22:53:40.382 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:40.387 INFO:teuthology.orchestra.run.smithi076.stdout:944 punch from 0x96e9094 to 0x96f04b8, (0x7424 bytes) 2024-09-14T22:53:40.387 INFO:teuthology.orchestra.run.smithi076.stdout:945 read 0x4e7a9c6 thru 0x4e88ff7 (0xe632 bytes) 2024-09-14T22:53:40.387 INFO:teuthology.orchestra.run.smithi076.stdout:946 read 0xa6c9e7b thru 0xa6d0733 (0x68b9 bytes) 2024-09-14T22:53:40.387 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.387 INFO:teuthology.orchestra.run.smithi076.stdout:948 read 0x8c13313 thru 0x8c1cb92 (0x9880 bytes) 2024-09-14T22:53:40.387 INFO:teuthology.orchestra.run.smithi076.stdout:949 punch from 0x3961162 to 0x3969824, (0x86c2 bytes) 2024-09-14T22:53:40.387 INFO:teuthology.orchestra.run.smithi076.stdout:950 read 0x8e6505 thru 0x8ef231 (0x8d2d bytes) 2024-09-14T22:53:40.388 INFO:teuthology.orchestra.run.smithi076.stdout:951 read 0x45b419b thru 0x45be296 (0xa0fc bytes) 2024-09-14T22:53:40.389 INFO:teuthology.orchestra.run.smithi076.stdout:952 trunc from 0xceeae31 to 0xd293fc2 2024-09-14T22:53:40.392 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.396 INFO:teuthology.orchestra.run.smithi076.stdout:954 trunc from 0xd293fc2 to 0xa466864 2024-09-14T22:53:40.402 INFO:teuthology.orchestra.run.smithi076.stdout:955 read 0x41c8479 thru 0x41d5677 (0xd1ff bytes) 2024-09-14T22:53:40.402 INFO:teuthology.orchestra.run.smithi076.stdout:956 read 0x86f24dc thru 0x86ff98a (0xd4af bytes) 2024-09-14T22:53:40.402 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.402 INFO:teuthology.orchestra.run.smithi076.stdout:959 punch from 0x6b24b87 to 0x6b28320, (0x3799 bytes) 2024-09-14T22:53:40.402 INFO:teuthology.orchestra.run.smithi076.stdout:960 writesame 0x931b47e thru 0x9325601 data_size 0x2(0xa184 bytes) 2024-09-14T22:53:40.413 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:40.412+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:40.421 INFO:teuthology.orchestra.run.smithi076.stdout:961 writesame 0xad911f8 thru 0xad95328 data_size 0x1(0x4131 bytes) 2024-09-14T22:53:40.421 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.421 INFO:teuthology.orchestra.run.smithi076.stdout:964 punch from 0x7ebd12c to 0x7ec0b0e, (0x39e2 bytes) 2024-09-14T22:53:40.433 INFO:teuthology.orchestra.run.smithi076.stdout:965 trunc from 0xad95329 to 0xe2e3a3 2024-09-14T22:53:40.444 INFO:teuthology.orchestra.run.smithi076.stdout:966 punch from 0x800ce8 to 0x8084ec, (0x7804 bytes) 2024-09-14T22:53:40.451 INFO:teuthology.orchestra.run.smithi076.stdout:967 trunc from 0xe2e3a3 to 0x698df69 2024-09-14T22:53:40.465 INFO:teuthology.orchestra.run.smithi076.stdout:969 write 0xa89efd3 thru 0xa8a2fd7 (0x4005 bytes) 2024-09-14T22:53:40.469 INFO:teuthology.orchestra.run.smithi076.stdout:971 read 0x8df7e76 thru 0x8e06941 (0xeacc bytes) 2024-09-14T22:53:40.469 INFO:teuthology.orchestra.run.smithi076.stdout:972 read 0x91c7b6a thru 0x91c9549 (0x19e0 bytes) 2024-09-14T22:53:40.469 INFO:teuthology.orchestra.run.smithi076.stdout:973 write 0xb1dea4 thru 0xb2bb29 (0xdc86 bytes) 2024-09-14T22:53:40.471 INFO:teuthology.orchestra.run.smithi076.stdout:975 read 0x8574143 thru 0x857462b (0x4e9 bytes) 2024-09-14T22:53:40.472 INFO:teuthology.orchestra.run.smithi076.stdout:977 read 0x5486581 thru 0x54887cb (0x224b bytes) 2024-09-14T22:53:40.472 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.480 INFO:teuthology.orchestra.run.smithi076.stdout:979 trunc from 0xa8a2fd8 to 0x393cec9 2024-09-14T22:53:40.488 INFO:teuthology.orchestra.run.smithi076.stdout:980 writesame 0x2d2aa9d thru 0x2d2bc81 data_size 0x3(0x11e5 bytes) 2024-09-14T22:53:40.502 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:40.502 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:40.502 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:40.502 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:40.502 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:40.502 INFO:teuthology.orchestra.run.smithi076.stdout:981 write 0x836aa73 thru 0x8371f11 (0x749f bytes) 2024-09-14T22:53:40.505 INFO:teuthology.orchestra.run.smithi076.stdout:983 read 0x3b36905 thru 0x3b42455 (0xbb51 bytes) 2024-09-14T22:53:40.513 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:40.513 INFO:teuthology.orchestra.run.smithi076.stdout:986 trunc from 0x8371f12 to 0x2005961 2024-09-14T22:53:40.514 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:40.513+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:987 writesame 0xad69070 thru 0xad73852 data_size 0x1(0xa7e3 bytes) 2024-09-14T22:53:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:988 read 0x8472f0c thru 0x847f740 (0xc835 bytes) 2024-09-14T22:53:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.539 INFO:teuthology.orchestra.run.smithi076.stdout:990 read 0x82406c0 thru 0x82461bb (0x5afc bytes) 2024-09-14T22:53:40.544 INFO:teuthology.orchestra.run.smithi076.stdout:993 write 0xb393d5f thru 0xb39cab5 (0x8d57 bytes) 2024-09-14T22:53:40.547 INFO:teuthology.orchestra.run.smithi076.stdout:994 writesame 0x5d23d4b thru 0x5d2c679 data_size 0x7(0x892f bytes) 2024-09-14T22:53:40.552 INFO:teuthology.orchestra.run.smithi076.stdout:995 read 0x4a11abc thru 0x4a159db (0x3f20 bytes) 2024-09-14T22:53:40.554 INFO:teuthology.orchestra.run.smithi076.stdout:996 trunc from 0xb39cab6 to 0x9a4d4f6 2024-09-14T22:53:40.564 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.564 INFO:teuthology.orchestra.run.smithi076.stdout:999 write 0x27600a6 thru 0x2767cf2 (0x7c4d bytes) 2024-09-14T22:53:40.572 INFO:teuthology.orchestra.run.smithi076.stdout:1000 trunc from 0x9a4d4f6 to 0x41c252c 2024-09-14T22:53:40.583 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:40.583 INFO:teuthology.orchestra.run.smithi076.stdout:1005 writesame 0x3feadf0 thru 0x3feea86 data_size 0x1(0x3c97 bytes) 2024-09-14T22:53:40.593 INFO:teuthology.orchestra.run.smithi076.stdout:1006 write 0x2b1c1fb thru 0x2b1fb9f (0x39a5 bytes) 2024-09-14T22:53:40.597 INFO:teuthology.orchestra.run.smithi076.stdout:1008 read 0xae9f13 thru 0xaed850 (0x393e bytes) 2024-09-14T22:53:40.597 INFO:teuthology.orchestra.run.smithi076.stdout:1009 clone 13 order 21 su 65536 sc 7 2024-09-14T22:53:40.616 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:40.614+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:40.648 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6497396630000001, 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:53:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:53:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6497206250000001, 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:53:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.649684669, 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.649663549, 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:53:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6484566350000001, 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:53:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6466271880000001, 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:53:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:53:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6460729010000001, 2024-09-14T22:53:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:53:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.644088183, 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:53:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6437358629999999, 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:53:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.64345242, 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:53:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:53:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6431643230000001, 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:53:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6427894950000002, 2024-09-14T22:53:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6424686799999999, 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:53:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6287834449999998, 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6284486359999999, 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:53:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:53:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6281184739999999, 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:53:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.627897688, 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:53:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:53:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6276283870000001, 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:53:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:53:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:53:40.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6274412910000002, 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:53:40.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.6271231319999999, 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:40.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:40.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:40.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:40.740 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:40.741 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:40.742 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:40.740+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:40.842 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:40.840+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:40.885 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:40.885 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:40.885 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:40.886 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:40.896 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:40.942 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:40.941+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:41.004 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:41.004 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:41.004 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:41.005 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:41.005 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:41.011 INFO:teuthology.orchestra.run.smithi076.stdout:leaving image image_client.0-clone12 intact 2024-09-14T22:53:41.014 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:41.043 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:41.042+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:41.123 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4232.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 43.590369609, 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010310849999999999, 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.124 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:41.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8500000000000002e-07 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530823+0000", 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0277e-05 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530835+0000", 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1459e-05 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530845+0000", 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0239e-05 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530907+0000", 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1981000000000002e-05 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530934+0000", 2024-09-14T22:53:41.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6701999999999999e-05 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.531810+0000", 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00087597399999999996 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.531844+0000", 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4068e-05 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8992 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'23, mlcod=33'21)", 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.214808112, 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0035453559999999999, 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:41.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906376+0000", 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.42e-06 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906968+0000", 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00059252399999999998 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906990+0000", 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1498999999999999e-05 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906999+0000", 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2299999999999997e-06 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907058+0000", 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9505e-05 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907082+0000", 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3298999999999998e-05 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909874+0000", 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027926689999999998 2024-09-14T22:53:41.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909919+0000", 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5210000000000003e-05 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8998 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'25, mlcod=33'21)", 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.213140178, 2024-09-14T22:53:41.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003268272, 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908043+0000", 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908054+0000", 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1654999999999999e-05 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908066+0000", 2024-09-14T22:53:41.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1321000000000001e-05 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908077+0000", 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1781e-05 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908149+0000", 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1402000000000005e-05 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908170+0000", 2024-09-14T22:53:41.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0746000000000001e-05 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911259+0000", 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030889839999999999 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911310+0000", 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1684999999999997e-05 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.137 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "pg_update_log_missing(3.9 epoch 34/24 rep_tid 166 entries 34'26 (0'0) error 3:94c3b88b:::rbd_data.113535cec067.000000000000010d:head by client.4405.0:10843 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 34'23)", 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.114828+0000", 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.006354369, 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00099293600000000004, 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114828+0000", 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114828+0000", 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114829+0000", 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:53:41.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114843+0000", 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3687e-05 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114871+0000", 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8269e-05 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114888+0000", 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.719e-05 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114980+0000", 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1689999999999998e-05 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.115821+0000", 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00084082199999999997 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.141 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15153 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'58, mlcod=34'56)", 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 22.41096576, 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043627650000000002, 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710218+0000", 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.291e-06 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710957+0000", 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00073905599999999998 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710977+0000", 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0506999999999999e-05 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710986+0000", 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0860000000000008e-06 2024-09-14T22:53:41.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711048+0000", 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1130999999999995e-05 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711072+0000", 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4167000000000001e-05 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714533+0000", 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034611070000000002 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714579+0000", 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6419999999999999e-05 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15154 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'60, mlcod=34'56)", 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:41.146 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 22.409615004999999, 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00342618, 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711568+0000", 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0900000000000002e-07 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711574+0000", 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5119999999999997e-06 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711583+0000", 2024-09-14T22:53:41.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7579999999999998e-06 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711589+0000", 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9619999999999998e-06 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711692+0000", 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000103237 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711719+0000", 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7124999999999999e-05 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714953+0000", 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032341240000000001 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714993+0000", 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9953000000000002e-05 2024-09-14T22:53:41.150 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77119 3.9 e40/24 3:9667b220:::rbd_data.113564f98e71.0000000000000024:head v 40'54, mlcod=39'51)", 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:26.993504+0000", 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.127677802000001, 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010398370000000001, 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993504+0000", 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993504+0000", 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993505+0000", 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993511+0000", 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2179999999999996e-06 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993529+0000", 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8280000000000001e-05 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993539+0000", 2024-09-14T22:53:41.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7820000000000005e-06 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993592+0000", 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3183999999999998e-05 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993616+0000", 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4043999999999999e-05 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.994500+0000", 2024-09-14T22:53:41.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00088420199999999995 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.994544+0000", 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3779999999999998e-05 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77339 3.9 e40/24 3:939e5cf2:::rbd_data.113564f98e71.0000000000000044:head v 40'60, mlcod=40'58)", 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.088999238, 2024-09-14T22:53:41.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0019342230000000001, 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032184+0000", 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032626+0000", 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000441369 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032644+0000", 2024-09-14T22:53:41.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8029999999999998e-05 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032652+0000", 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7760000000000003e-06 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032720+0000", 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032743+0000", 2024-09-14T22:53:41.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2823000000000002e-05 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034080+0000", 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001337705 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034117+0000", 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6854000000000003e-05 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77340 3.9 e40/24 3:939e5cf2:::rbd_data.113564f98e71.0000000000000044:head v 40'62, mlcod=40'58)", 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.088110196000001, 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00147386, 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033073+0000", 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9999999999999998e-07 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033078+0000", 2024-09-14T22:53:41.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5430000000000001e-06 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033087+0000", 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5420000000000005e-06 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033093+0000", 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9479999999999996e-06 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033185+0000", 2024-09-14T22:53:41.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2298000000000003e-05 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033218+0000", 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3389999999999997e-05 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034501+0000", 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012830160000000001 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034546+0000", 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4623000000000003e-05 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77389 3.9 e40/24 3:95bf6a49:::rbd_data.113564f98e71.0000000000000053:head v 40'64, mlcod=40'60)", 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.040338+0000", 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.080844697, 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0009955369999999999, 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.164 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040338+0000", 2024-09-14T22:53:41.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040338+0000", 2024-09-14T22:53:41.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040339+0000", 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.403e-06 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040348+0000", 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2329999999999995e-06 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040364+0000", 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6137000000000002e-05 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040372+0000", 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1890000000000007e-06 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040435+0000", 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2416999999999995e-05 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040456+0000", 2024-09-14T22:53:41.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1058000000000001e-05 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.041290+0000", 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00083392600000000002 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.041333+0000", 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3173999999999997e-05 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 13.912660766, 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:53:41.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:78175 3.4 e41/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 41'105, mlcod=40'102)", 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:41.176 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 13.155381812, 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0049595489999999997, 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965801+0000", 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.35e-07 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965834+0000", 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3683999999999998e-05 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965846+0000", 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1620999999999999e-05 2024-09-14T22:53:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965855+0000", 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1130000000000007e-06 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965939+0000", 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4029000000000005e-05 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965963+0000", 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3529000000000001e-05 2024-09-14T22:53:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970705+0000", 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.004742853 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970760+0000", 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4484999999999998e-05 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.180 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPush(3.1e 45/44 [PushOp(3:7c4423e3:::rbd_data.113564f98e71.0000000000000010:b, version: 40'64, data_included: [0~12288,14336~22528,38912~22528,63488~198656], data_size: 256000, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7c4423e3:::rbd_data.113564f98e71.0000000000000010:b@40'64, size: 262144, copy_subset: [(0, 262144)], clone_subset: {}, snapset: b=[]:{b: [b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262144, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:32.810280+0000", 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.3109017810000001, 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010089669999999999, 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810280+0000", 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810280+0000", 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810281+0000", 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810505+0000", 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022378099999999999 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810522+0000", 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6693000000000001e-05 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810539+0000", 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7309e-05 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810629+0000", 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9928999999999999e-05 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810643+0000", 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4277000000000001e-05 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.811289+0000", 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00064626699999999996 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:114231 3.1e e48/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 48'96, mlcod=46'94)", 2024-09-14T22:53:41.184 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:35.958526+0000", 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.1626560570000004, 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00096878600000000002, 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.958526+0000", 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.958526+0000", 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.958527+0000", 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6599999999999994e-07 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.958536+0000", 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3030000000000005e-06 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.958548+0000", 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2047e-05 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.958557+0000", 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0639999999999994e-06 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.958619+0000", 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2123000000000001e-05 2024-09-14T22:53:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.958642+0000", 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3434e-05 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.959457+0000", 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00081488700000000003 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.959495+0000", 2024-09-14T22:53:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7962e-05 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:114233 3.1e e48/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 48'97, mlcod=46'95)", 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:35.966011+0000", 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.1551708889999999, 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00096213099999999997, 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966011+0000", 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966011+0000", 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966013+0000", 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5069999999999999e-06 2024-09-14T22:53:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966025+0000", 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2334000000000001e-05 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966044+0000", 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8862999999999999e-05 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966071+0000", 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7080000000000002e-05 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966135+0000", 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4166999999999997e-05 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966161+0000", 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5541000000000002e-05 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966932+0000", 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00077133799999999999 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:35.966973+0000", 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1301000000000003e-05 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:116229 3.1e e48/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 48'99, mlcod=48'97)", 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.8170899110000001, 2024-09-14T22:53:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034125309999999999, 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304095+0000", 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3029999999999998e-06 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304126+0000", 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1374999999999997e-05 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304140+0000", 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4518000000000001e-05 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304150+0000", 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304213+0000", 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3366000000000003e-05 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304237+0000", 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4379000000000002e-05 2024-09-14T22:53:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307471+0000", 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032338829999999999 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307505+0000", 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3552e-05 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:121888 3.1e e49/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 49'101, mlcod=48'99)", 2024-09-14T22:53:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.919965+0000", 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.2012168320000001, 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00092317699999999996, 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.919965+0000", 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.919965+0000", 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.919966+0000", 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0900000000000001e-07 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.919971+0000", 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.919986+0000", 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.56e-05 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.919996+0000", 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3400000000000004e-06 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.920058+0000", 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2175999999999996e-05 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.920076+0000", 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7968000000000001e-05 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.920854+0000", 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00077826999999999996 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.920889+0000", 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4613000000000002e-05 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.78899899200000001, 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:53:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:131041 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'121, mlcod=52'119)", 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.469601+0000", 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.65158153500000004, 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012083899999999999, 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469601+0000", 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469601+0000", 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469602+0000", 2024-09-14T22:53:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2309999999999999e-06 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469641+0000", 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8751000000000002e-05 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469655+0000", 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4299e-05 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469661+0000", 2024-09-14T22:53:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5470000000000002e-06 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469706+0000", 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4747999999999999e-05 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469722+0000", 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.470782+0000", 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001060273 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.470809+0000", 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7254e-05 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:131066 3.1e e52/44 3:7b98fc53:::rbd_data.1135a9fbf9fc.000000000000000f:head v 52'143, mlcod=52'141)", 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.503635+0000", 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.61754730599999996, 2024-09-14T22:53:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010888740000000001, 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503635+0000", 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503635+0000", 2024-09-14T22:53:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503637+0000", 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.84e-06 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503700+0000", 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3157000000000005e-05 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503718+0000", 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503743+0000", 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5208999999999999e-05 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503822+0000", 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9328999999999999e-05 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503847+0000", 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4624e-05 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.504687+0000", 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000840032 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.504724+0000", 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6689000000000001e-05 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.215 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:41.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:41.217 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T22:53:41.217 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:41.215+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:41.317 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:41.316+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:41.341 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:41.341 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:41.341 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:41.341 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:41.351 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:41.417 INFO:teuthology.orchestra.run.smithi076.stdout:1010 punch from 0x2db4443 to 0x2dbaab9, (0x6676 bytes) 2024-09-14T22:53:41.418 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:41.417+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:41.422 INFO:teuthology.orchestra.run.smithi076.stdout:1011 read 0x2d24b3 thru 0x2da521 (0x806f bytes) 2024-09-14T22:53:41.423 INFO:teuthology.orchestra.run.smithi076.stdout:1012 trunc from 0x41c252c to 0x44ebc7d 2024-09-14T22:53:41.430 INFO:teuthology.orchestra.run.smithi076.stdout:1014 read 0x44891dd thru 0x449055e (0x7382 bytes) 2024-09-14T22:53:41.440 INFO:teuthology.orchestra.run.smithi076.stdout:1015 trunc from 0x44ebc7d to 0xbad697b 2024-09-14T22:53:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:1016 punch from 0x5a43314 to 0x5a4eb34, (0xb820 bytes) 2024-09-14T22:53:41.446 INFO:teuthology.orchestra.run.smithi076.stdout:1017 write 0x98314d3 thru 0x98348e8 (0x3416 bytes) 2024-09-14T22:53:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:1018 read 0x87f065f thru 0x87f879b (0x813d bytes) 2024-09-14T22:53:41.449 INFO:teuthology.orchestra.run.smithi076.stdout:1020 read 0x4698210 thru 0x46a2c64 (0xaa55 bytes) 2024-09-14T22:53:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:1021 punch from 0xa201208 to 0xa207cf7, (0x6aef bytes) 2024-09-14T22:53:41.450 INFO:teuthology.orchestra.run.smithi076.stdout:1022 writesame 0xa8b0a3 thru 0xa90941 data_size 0x7(0x589f bytes) 2024-09-14T22:53:41.469 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:41.469 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:41.469 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:41.469 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:41.469 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:41.478 INFO:teuthology.orchestra.run.smithi076.stdout:1023 read 0x9af77e1 thru 0x9b01840 (0xa060 bytes) 2024-09-14T22:53:41.478 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:41.478 INFO:teuthology.orchestra.run.smithi076.stdout:1029 read 0x9dee7da thru 0x9df2f04 (0x472b bytes) 2024-09-14T22:53:41.478 INFO:teuthology.orchestra.run.smithi076.stdout:1030 write 0xd1ac42 thru 0xd277e0 (0xcb9f bytes) 2024-09-14T22:53:41.479 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:41.496 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:41.496 INFO:teuthology.orchestra.run.smithi076.stdout:1032 read 0x48952bb thru 0x4896840 (0x1586 bytes) 2024-09-14T22:53:41.503 INFO:teuthology.orchestra.run.smithi076.stdout:1033 write 0xda6ee84 thru 0xda7c405 (0xd582 bytes) 2024-09-14T22:53:41.507 INFO:teuthology.orchestra.run.smithi076.stdout:1034 write 0xbf40c08 thru 0xbf438b7 (0x2cb0 bytes) 2024-09-14T22:53:41.510 INFO:teuthology.orchestra.run.smithi076.stdout:1035 punch from 0x229c282 to 0x22a23e0, (0x615e bytes) 2024-09-14T22:53:41.510 INFO:teuthology.orchestra.run.smithi076.stdout:1036 writesame 0x4f6f178 thru 0x4f75a53 data_size 0x6(0x68dc bytes) 2024-09-14T22:53:41.516 INFO:teuthology.orchestra.run.smithi076.stdout:1037 write 0x7a00842 thru 0x7a0d677 (0xce36 bytes) 2024-09-14T22:53:41.518 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:41.517+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:41.520 INFO:teuthology.orchestra.run.smithi076.stdout:1038 trunc from 0xda7c406 to 0xe45dd44 2024-09-14T22:53:41.525 INFO:teuthology.orchestra.run.smithi076.stdout:1039 writesame 0x8ba791d thru 0x8bb6c36 data_size 0x3a(0xf31a bytes) 2024-09-14T22:53:41.537 INFO:teuthology.orchestra.run.smithi076.stdout:1040 trunc from 0xe45dd44 to 0x844d6a8 2024-09-14T22:53:41.560 INFO:teuthology.orchestra.run.smithi076.stdout:1041 write 0xad115e8 thru 0xad1ffc7 (0xe9e0 bytes) 2024-09-14T22:53:41.563 INFO:teuthology.orchestra.run.smithi076.stdout:1042 write 0x9499c1a thru 0x94a7343 (0xd72a bytes) 2024-09-14T22:53:41.568 INFO:teuthology.orchestra.run.smithi076.stdout:1043 write 0x6c69350 thru 0x6c6f057 (0x5d08 bytes) 2024-09-14T22:53:41.573 INFO:teuthology.orchestra.run.smithi076.stdout:1044 read 0x8358a06 thru 0x8359947 (0xf42 bytes) 2024-09-14T22:53:41.573 INFO:teuthology.orchestra.run.smithi076.stdout:1046 read 0x81cd0c0 thru 0x81ce621 (0x1562 bytes) 2024-09-14T22:53:41.573 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:41.573 INFO:teuthology.orchestra.run.smithi076.stdout:1048 flatten 2024-09-14T22:53:41.597 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:41.597 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:41.597 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:41.597 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30305 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1349632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690739688000001, 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051104002000000003, 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30305 2024-09-14T22:53:41.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:41.599 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8799999999999998e-07 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904388+0000", 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7000000000000008e-06 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904395+0000", 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904401+0000", 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.147e-06 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955309+0000", 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.050908627999999997 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955388+0000", 2024-09-14T22:53:41.601 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8615999999999994e-05 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955484+0000", 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6323000000000006e-05 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690712177999998, 2024-09-14T22:53:41.602 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:53:41.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.605 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.606 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690691642000001, 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.607 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:53:41.608 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.609 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.610 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690661642999999, 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:53:41.611 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:53:41.613 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:53:41.615 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.69063049, 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.616 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.617 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:53:41.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690607834000001, 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:53:41.620 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.622 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:53:41.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.624 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690587333, 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:53:41.625 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:53:41.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.629 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690567571999999, 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.630 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:53:41.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:53:41.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690546786999999, 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.634 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:53:41.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690514374999999, 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:53:41.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:53:41.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.643 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690492976000002, 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:53:41.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:53:41.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690431576999998, 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.648 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:53:41.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.652 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690391664, 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.653 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:53:41.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:53:41.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690361908, 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:53:41.657 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.658 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.661 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690331672999999, 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:41.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.663 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:53:41.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690306933999999, 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.666 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:53:41.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.670 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690283504, 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690187768000001, 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:53:41.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:53:41.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690158845999999, 2024-09-14T22:53:41.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:41.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:53:41.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:53:41.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.684 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.690135520999998, 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:41.685 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:53:41.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:41.689 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:41.691 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:41.692 INFO:teuthology.orchestra.run.smithi076.stdout:1049 punch from 0xb03016 to 0xb09ec0, (0x6eaa bytes) 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1051 punch from 0x29067f6 to 0x2914e01, (0xe60b bytes) 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1053 read 0x59c590a thru 0x59d32be (0xd9b5 bytes) 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1054 trunc from 0xad1ffc8 to 0x5f0ab53 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1055 writesame 0xe63a58d thru 0xe6457a4 data_size 0x8b(0xb218 bytes) 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1057 write 0x9818499 thru 0x9824ee5 (0xca4d bytes) 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1058 trunc from 0xe6457a5 to 0xacd09c3 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1060 punch from 0xa022127 to 0xa02b470, (0x9349 bytes) 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1062 writesame 0xde94523 thru 0xde9abeb data_size 0x31(0x66c9 bytes) 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1064 punch from 0x3c3e286 to 0x3c41eff, (0x3c79 bytes) 2024-09-14T22:53:41.693 INFO:teuthology.orchestra.run.smithi076.stdout:1065 punch from 0x5ad6688 to 0x5ad8ed3, (0x284b bytes) 2024-09-14T22:53:41.694 INFO:teuthology.orchestra.run.smithi076.stdout:1066 writesame 0x9e7131f thru 0x9e72ddb data_size 0x5(0x1abd bytes) 2024-09-14T22:53:41.694 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:41.694 INFO:teuthology.orchestra.run.smithi076.stdout:1068 writesame 0x2ecd705 thru 0x2ed2dee data_size 0x59(0x56ea bytes) 2024-09-14T22:53:41.694 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:41.689+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:41.695 INFO:teuthology.orchestra.run.smithi076.stdout:1069 read 0x124ab95 thru 0x124aede (0x34a bytes) 2024-09-14T22:53:41.695 INFO:teuthology.orchestra.run.smithi076.stdout:1070 write 0x1a4f2f thru 0x1a5ec9 (0xf9b bytes) 2024-09-14T22:53:41.695 INFO:teuthology.orchestra.run.smithi076.stdout:1071 write 0xc4fe108 thru 0xc50d46b (0xf364 bytes) 2024-09-14T22:53:41.699 INFO:teuthology.orchestra.run.smithi076.stdout:1072 read 0xc2fedc5 thru 0xc30576f (0x69ab bytes) 2024-09-14T22:53:41.702 INFO:teuthology.orchestra.run.smithi076.stdout:1073 trunc from 0xde9abec to 0xbee13fb 2024-09-14T22:53:41.711 INFO:teuthology.orchestra.run.smithi076.stdout:1074 write 0x9394469 thru 0x939b608 (0x71a0 bytes) 2024-09-14T22:53:41.714 INFO:teuthology.orchestra.run.smithi076.stdout:1075 write 0xb2d331d thru 0xb2d7042 (0x3d26 bytes) 2024-09-14T22:53:41.717 INFO:teuthology.orchestra.run.smithi076.stdout:1076 write 0xa3a4e4a thru 0xa3af425 (0xa5dc bytes) 2024-09-14T22:53:41.720 INFO:teuthology.orchestra.run.smithi076.stdout:1077 read 0x7338821 thru 0x73483b9 (0xfb99 bytes) 2024-09-14T22:53:41.720 INFO:teuthology.orchestra.run.smithi076.stdout:1078 write 0x76cfeeb thru 0x76d0db3 (0xec9 bytes) 2024-09-14T22:53:41.726 INFO:teuthology.orchestra.run.smithi076.stdout:1079 read 0x97c3f7c thru 0x97c82bb (0x4340 bytes) 2024-09-14T22:53:41.727 INFO:teuthology.orchestra.run.smithi076.stdout:1080 read 0xf5de54 thru 0xf69186 (0xb333 bytes) 2024-09-14T22:53:41.727 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:41.734 INFO:teuthology.orchestra.run.smithi076.stdout:1082 write 0xcfb3ed7 thru 0xcfc19f4 (0xdb1e bytes) 2024-09-14T22:53:41.737 INFO:teuthology.orchestra.run.smithi076.stdout:1084 read 0xbce6062 thru 0xbce9c8e (0x3c2d bytes) 2024-09-14T22:53:41.743 INFO:teuthology.orchestra.run.smithi076.stdout:1085 write 0xd92893c thru 0xd936259 (0xd91e bytes) 2024-09-14T22:53:41.748 INFO:teuthology.orchestra.run.smithi076.stdout:1086 read 0x998b6f1 thru 0x998b886 (0x196 bytes) 2024-09-14T22:53:41.748 INFO:teuthology.orchestra.run.smithi076.stdout:1087 clone 14 order 25 su 131072 sc 14 2024-09-14T22:53:41.791 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:41.789+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:41.804 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:41.804 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:41.804 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:41.804 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:41.812 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:41.892 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:41.889+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:41.902 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:53:41.920 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:41.920 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:41.920 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:41.920 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:41.920 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:41.929 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:41.993 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:41.991+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116113 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5611520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9789353900000002, 2024-09-14T22:53:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053031104000000003, 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116113 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053986+0000", 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010658 2024-09-14T22:53:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053989+0000", 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3579999999999999e-06 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053998+0000", 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8319999999999995e-06 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054004+0000", 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3579999999999999e-06 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106732+0000", 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052727505000000001 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106816+0000", 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4482999999999997e-05 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106910+0000", 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3987999999999995e-05 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9787996919999999, 2024-09-14T22:53:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:53:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:53:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9786045999999997, 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:53:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9784518100000001, 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9782467879999999, 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9779761220000003, 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.058 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:53:42.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9777139620000002, 2024-09-14T22:53:42.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:53:42.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:53:42.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9775551099999999, 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.070 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9773683139999996, 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:53:42.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:53:42.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9771660689999999, 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:53:42.075 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:53:42.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:53:42.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.079 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9769577109999998, 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:53:42.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:53:42.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:53:42.083 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9766439939999998, 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.084 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:53:42.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:53:42.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9764382319999996, 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.088 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9762693559999995, 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:53:42.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:53:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9759986620000003, 2024-09-14T22:53:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:53:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:53:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9758493870000002, 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:53:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9756081449999998, 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:53:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:53:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116130 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5566464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9753871109999999, 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053025477000000001, 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116130 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057631+0000", 2024-09-14T22:53:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020390499999999999 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057636+0000", 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4299999999999999e-06 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057647+0000", 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1516e-05 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057653+0000", 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3620000000000002e-06 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110336+0000", 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052682584999999997 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110379+0000", 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2901000000000001e-05 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110453+0000", 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3777999999999997e-05 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116131 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5685248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9751495979999998, 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053011547999999999, 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116131 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057795+0000", 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013012700000000001 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057805+0000", 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0482999999999999e-05 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057817+0000", 2024-09-14T22:53:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1499e-05 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057827+0000", 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0091999999999999e-05 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110493+0000", 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052666339999999999 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110578+0000", 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4876000000000005e-05 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110676+0000", 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8130999999999999e-05 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:53:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6669181130000004, 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:53:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:53:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:42.125 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:42.125 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:42.127 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:42.124+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:42.127 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:42.145 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T22:53:42.228 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:42.226+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:42.237 INFO:teuthology.orchestra.run.smithi076.stdout:1088 read 0x8527cdd thru 0x852b85f (0x3b83 bytes) 2024-09-14T22:53:42.238 INFO:teuthology.orchestra.run.smithi076.stdout:1090 read 0xbcb84d6 thru 0xbcbb347 (0x2e72 bytes) 2024-09-14T22:53:42.239 INFO:teuthology.orchestra.run.smithi076.stdout:1091 trunc from 0xd93625a to 0xe109bd9 2024-09-14T22:53:42.248 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:42.248 INFO:teuthology.orchestra.run.smithi076.stdout:1093 read 0x26407d5 thru 0x2644295 (0x3ac1 bytes) 2024-09-14T22:53:42.248 INFO:teuthology.orchestra.run.smithi076.stdout:1094 write 0x90d8106 thru 0x90d9e26 (0x1d21 bytes) 2024-09-14T22:53:42.259 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:42.261 INFO:teuthology.orchestra.run.smithi076.stdout:1095 punch from 0x90a90ac to 0x90b417c, (0xb0d0 bytes) 2024-09-14T22:53:42.261 INFO:teuthology.orchestra.run.smithi076.stdout:1096 write 0x6ae89d1 thru 0x6af303c (0xa66c bytes) 2024-09-14T22:53:42.273 INFO:teuthology.orchestra.run.smithi076.stdout:1098 read 0x52f5f91 thru 0x52f61a8 (0x218 bytes) 2024-09-14T22:53:42.273 INFO:teuthology.orchestra.run.smithi076.stdout:1099 write 0x12e962c thru 0x12f30d8 (0x9aad bytes) 2024-09-14T22:53:42.279 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:42.279 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:42.279 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:42.279 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:42.280 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:42.280 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:42.284 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:42.284 INFO:teuthology.orchestra.run.smithi076.stdout:1101 write 0xb9e1257 thru 0xb9efdeb (0xeb95 bytes) 2024-09-14T22:53:42.290 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:42.298 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:42.298 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:42.304 INFO:teuthology.orchestra.run.smithi076.stdout:1102 trunc from 0xe109bd9 to 0x4800c03 2024-09-14T22:53:42.322 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:42.322 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:42.332 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:42.330+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:42.345 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:42.348 INFO:teuthology.orchestra.run.smithi076.stdout:1103 write 0xd2d18 thru 0xdb45b (0x8744 bytes) 2024-09-14T22:53:42.350 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:42.350 INFO:teuthology.orchestra.run.smithi076.stdout:1107 read 0x30ab79e thru 0x30b58d9 (0xa13c bytes) 2024-09-14T22:53:42.363 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:42.367 INFO:teuthology.orchestra.run.smithi076.stdout:1108 writesame 0xcaa44b2 thru 0xcab1946 data_size 0x1(0xd495 bytes) 2024-09-14T22:53:42.368 INFO:teuthology.orchestra.run.smithi076.stdout:1109 writesame 0x55b6839 thru 0x55c0c73 data_size 0x1(0xa43b bytes) 2024-09-14T22:53:42.381 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:42.396 INFO:teuthology.orchestra.run.smithi076.stdout:1110 write 0xad009b thru 0xad0e46 (0xdac bytes) 2024-09-14T22:53:42.396 INFO:teuthology.orchestra.run.smithi076.stdout:1112 write 0xa370fb6 thru 0xa380093 (0xf0de bytes) 2024-09-14T22:53:42.398 INFO:teuthology.orchestra.run.smithi076.stdout:1113 read 0xbb3e1a5 thru 0xbb4afb8 (0xce14 bytes) 2024-09-14T22:53:42.399 INFO:teuthology.orchestra.run.smithi076.stdout:1115 write 0x17448e thru 0x17f1db (0xad4e bytes) 2024-09-14T22:53:42.399 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:42.400 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:42.401 INFO:teuthology.orchestra.run.smithi076.stdout:1116 read 0x888edfc thru 0x8891a86 (0x2c8b bytes) 2024-09-14T22:53:42.401 INFO:teuthology.orchestra.run.smithi076.stdout:1117 write 0xb2ca063 thru 0xb2ce7e0 (0x477e bytes) 2024-09-14T22:53:42.403 INFO:teuthology.orchestra.run.smithi076.stdout:1118 read 0x5eef35 thru 0x5fd96e (0xea3a bytes) 2024-09-14T22:53:42.404 INFO:teuthology.orchestra.run.smithi076.stdout:1119 read 0xd57c7e thru 0xd6287a (0xabfd bytes) 2024-09-14T22:53:42.405 INFO:teuthology.orchestra.run.smithi076.stdout:1120 read 0x3592284 thru 0x359866a (0x63e7 bytes) 2024-09-14T22:53:42.405 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:42.417 INFO:teuthology.orchestra.run.smithi076.stdout:1123 trunc from 0xcab1947 to 0x2c31d51 2024-09-14T22:53:42.429 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:42.427+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:42.442 INFO:teuthology.orchestra.run.smithi076.stdout:1125 write 0x8b2ce0e thru 0x8b2ffd4 (0x31c7 bytes) 2024-09-14T22:53:42.443 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:42.443 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:42.444 INFO:teuthology.orchestra.run.smithi076.stdout:1126 read 0x3d1cebc thru 0x3d24e17 (0x7f5c bytes) 2024-09-14T22:53:42.450 INFO:teuthology.orchestra.run.smithi076.stdout:1127 trunc from 0x8b2ffd5 to 0x4178c27 2024-09-14T22:53:42.453 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:42.462 INFO:teuthology.orchestra.run.smithi076.stdout:1128 trunc from 0x4178c27 to 0x655c676 2024-09-14T22:53:42.469 INFO:teuthology.orchestra.run.smithi076.stdout:1130 write 0x4e0740b thru 0x4e0cfc8 (0x5bbe bytes) 2024-09-14T22:53:42.471 INFO:teuthology.orchestra.run.smithi076.stdout:1131 read 0x4e24ad8 thru 0x4e33d2f (0xf258 bytes) 2024-09-14T22:53:42.471 INFO:teuthology.orchestra.run.smithi076.stdout:1132 write 0x4766e52 thru 0x4766f09 (0xb8 bytes) 2024-09-14T22:53:42.473 INFO:teuthology.orchestra.run.smithi076.stdout:1133 read 0x5c750ef thru 0x5c8354f (0xe461 bytes) 2024-09-14T22:53:42.473 INFO:teuthology.orchestra.run.smithi076.stdout:1134 write 0x5fdbae3 thru 0x5fdfd91 (0x42af bytes) 2024-09-14T22:53:42.475 INFO:teuthology.orchestra.run.smithi076.stdout:1136 read 0x653899f thru 0x653c39a (0x39fc bytes) 2024-09-14T22:53:42.475 INFO:teuthology.orchestra.run.smithi076.stdout:1139 punch from 0x96d478 to 0x96d4b8, (0x40 bytes) 2024-09-14T22:53:42.488 INFO:teuthology.orchestra.run.smithi076.stdout:1141 write 0xc4f8fea thru 0xc4fe7f9 (0x5810 bytes) 2024-09-14T22:53:42.490 INFO:teuthology.orchestra.run.smithi076.stdout:1143 read 0xa8db8ce thru 0xa8e065d (0x4d90 bytes) 2024-09-14T22:53:42.490 INFO:teuthology.orchestra.run.smithi076.stdout:1144 writesame 0x2ba27a0 thru 0x2ba880e data_size 0x27(0x606f bytes) 2024-09-14T22:53:42.492 INFO:teuthology.orchestra.run.smithi076.stdout:1145 punch from 0x6df1840 to 0x6df8e03, (0x75c3 bytes) 2024-09-14T22:53:42.492 INFO:teuthology.orchestra.run.smithi076.stdout:1146 punch from 0x819b6b7 to 0x81a1ac5, (0x640e bytes) 2024-09-14T22:53:42.499 INFO:teuthology.orchestra.run.smithi076.stdout:1147 trunc from 0xc4fe7fa to 0x69eb6de 2024-09-14T22:53:42.520 INFO:teuthology.orchestra.run.smithi076.stdout:1149 write 0xbb4d1d0 thru 0xbb5aae3 (0xd914 bytes) 2024-09-14T22:53:42.530 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:42.528+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:42.530 INFO:teuthology.orchestra.run.smithi076.stdout:1152 writesame 0xdf8167f thru 0xdf82804 data_size 0x1(0x1186 bytes) 2024-09-14T22:53:42.531 INFO:teuthology.orchestra.run.smithi076.stdout:1153 write 0x7a550c7 thru 0x7a5aec7 (0x5e01 bytes) 2024-09-14T22:53:42.533 INFO:teuthology.orchestra.run.smithi076.stdout:1154 write 0x886def7 thru 0x88771b5 (0x92bf bytes) 2024-09-14T22:53:42.535 INFO:teuthology.orchestra.run.smithi076.stdout:1156 write 0x22e3459 thru 0x22e71e8 (0x3d90 bytes) 2024-09-14T22:53:42.536 INFO:teuthology.orchestra.run.smithi076.stdout:1157 read 0x368633e thru 0x3688a88 (0x274b bytes) 2024-09-14T22:53:42.541 INFO:teuthology.orchestra.run.smithi076.stdout:1158 writesame 0xe123309 thru 0xe12ef63 data_size 0x1(0xbc5b bytes) 2024-09-14T22:53:42.543 INFO:teuthology.orchestra.run.smithi076.stdout:1159 punch from 0xd2ccb8e to 0xd2d36b2, (0x6b24 bytes) 2024-09-14T22:53:42.543 INFO:teuthology.orchestra.run.smithi076.stdout:1160 read 0xdb18372 thru 0xdb24782 (0xc411 bytes) 2024-09-14T22:53:42.543 INFO:teuthology.orchestra.run.smithi076.stdout:1161 read 0xdfd5492 thru 0xdfdb6b9 (0x6228 bytes) 2024-09-14T22:53:42.543 INFO:teuthology.orchestra.run.smithi076.stdout:1162 write 0xd92d7d2 thru 0xd932e67 (0x5696 bytes) 2024-09-14T22:53:42.545 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:42.545 INFO:teuthology.orchestra.run.smithi076.stdout:1165 writesame 0x366844a thru 0x3676c9f data_size 0x6(0xe856 bytes) 2024-09-14T22:53:42.559 INFO:teuthology.orchestra.run.smithi076.stdout:1166 trunc from 0xe12ef64 to 0x74f65dd 2024-09-14T22:53:42.562 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:42.563 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:42.572 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:42.587 INFO:teuthology.orchestra.run.smithi076.stdout:1169 write 0xd7069ce thru 0xd70f532 (0x8b65 bytes) 2024-09-14T22:53:42.588 INFO:teuthology.orchestra.run.smithi076.stdout:1170 read 0x43fa085 thru 0x43fda34 (0x39b0 bytes) 2024-09-14T22:53:42.588 INFO:teuthology.orchestra.run.smithi076.stdout:1171 punch from 0x8f32323 to 0x8f4135e, (0xf03b bytes) 2024-09-14T22:53:42.604 INFO:teuthology.orchestra.run.smithi076.stdout:1173 trunc from 0xd70f533 to 0x392f6f 2024-09-14T22:53:42.618 INFO:teuthology.orchestra.run.smithi076.stdout:1174 read 0x12c31 thru 0x1e034 (0xb404 bytes) 2024-09-14T22:53:42.618 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:42.631 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:42.630+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:42.640 INFO:teuthology.orchestra.run.smithi076.stdout:1176 write 0xd0110fb thru 0xd0122dd (0x11e3 bytes) 2024-09-14T22:53:42.641 INFO:teuthology.orchestra.run.smithi076.stdout:1180 read 0xacfa43e thru 0xacffdc6 (0x5989 bytes) 2024-09-14T22:53:42.648 INFO:teuthology.orchestra.run.smithi076.stdout:1181 write 0xd6ae1ce thru 0xd6b53d4 (0x7207 bytes) 2024-09-14T22:53:42.650 INFO:teuthology.orchestra.run.smithi076.stdout:1182 read 0x4243721 thru 0x4251a1e (0xe2fe bytes) 2024-09-14T22:53:42.650 INFO:teuthology.orchestra.run.smithi076.stdout:1183 read 0x85b1476 thru 0x85b5a2b (0x45b6 bytes) 2024-09-14T22:53:42.651 INFO:teuthology.orchestra.run.smithi076.stdout:1186 writesame 0x9c6c7b4 thru 0x9c76e9d data_size 0x5(0xa6ea bytes) 2024-09-14T22:53:42.656 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:42.657 INFO:teuthology.orchestra.run.smithi076.stdout:1189 read 0xbb98923 thru 0xbb9e1c3 (0x58a1 bytes) 2024-09-14T22:53:42.657 INFO:teuthology.orchestra.run.smithi076.stdout:1191 punch from 0xc1044d8 to 0xc107d4e, (0x3876 bytes) 2024-09-14T22:53:42.668 INFO:teuthology.orchestra.run.smithi076.stdout:1193 trunc from 0xd6b53d5 to 0x3f92675 2024-09-14T22:53:42.682 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:42.682 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:42.685 INFO:teuthology.orchestra.run.smithi076.stdout:1194 write 0x6d8815e thru 0x6d8d5cb (0x546e bytes) 2024-09-14T22:53:42.691 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:42.695 INFO:teuthology.orchestra.run.smithi076.stdout:1195 trunc from 0x6d8d5cc to 0xe844af4 2024-09-14T22:53:42.702 INFO:teuthology.orchestra.run.smithi076.stdout:1196 punch from 0x4bf8e71 to 0x4c03b02, (0xac91 bytes) 2024-09-14T22:53:42.702 INFO:teuthology.orchestra.run.smithi076.stdout:1197 write 0x683a50a thru 0x683bc1a (0x1711 bytes) 2024-09-14T22:53:42.703 INFO:teuthology.orchestra.run.smithi076.stdout:1198 write 0x6f23df3 thru 0x6f24aa6 (0xcb4 bytes) 2024-09-14T22:53:42.704 INFO:teuthology.orchestra.run.smithi076.stdout:1199 writesame 0x82bfb85 thru 0x82ccf85 data_size 0x3(0xd401 bytes) 2024-09-14T22:53:42.715 INFO:teuthology.orchestra.run.smithi076.stdout:1200 writesame 0x54ba3ac thru 0x54c9b1c data_size 0xf(0xf771 bytes) 2024-09-14T22:53:42.719 INFO:teuthology.orchestra.run.smithi076.stdout:1201 punch from 0x60ccf98 to 0x60d2018, (0x5080 bytes) 2024-09-14T22:53:42.719 INFO:teuthology.orchestra.run.smithi076.stdout:1202 punch from 0x7cf1db4 to 0x7cf3c6a, (0x1eb6 bytes) 2024-09-14T22:53:42.719 INFO:teuthology.orchestra.run.smithi076.stdout:1203 punch from 0xc46f13a to 0xc4767b8, (0x767e bytes) 2024-09-14T22:53:42.719 INFO:teuthology.orchestra.run.smithi076.stdout:1206 read 0x4fbd029 thru 0x4fc67b7 (0x978f bytes) 2024-09-14T22:53:42.720 INFO:teuthology.orchestra.run.smithi076.stdout:1207 punch from 0x15e111d to 0x15e4a7c, (0x395f bytes) 2024-09-14T22:53:42.720 INFO:teuthology.orchestra.run.smithi076.stdout:1209 read 0xb6919cc thru 0xb69573c (0x3d71 bytes) 2024-09-14T22:53:42.720 INFO:teuthology.orchestra.run.smithi076.stdout:1210 writesame 0x4e5a1d5 thru 0x4e61a04 data_size 0x18(0x7830 bytes) 2024-09-14T22:53:42.722 INFO:teuthology.orchestra.run.smithi076.stdout:1211 read 0x6605165 thru 0x660d075 (0x7f11 bytes) 2024-09-14T22:53:42.722 INFO:teuthology.orchestra.run.smithi076.stdout:1212 write 0xff9f67 thru 0x1002829 (0x88c3 bytes) 2024-09-14T22:53:42.723 INFO:teuthology.orchestra.run.smithi076.stdout:1213 clone 15 order 25 su 16777216 sc 14 2024-09-14T22:53:42.733 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:42.731+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:42.834 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:42.831+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:42.835 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:42.835 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:42.844 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:42.886 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:42.886 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:42.886 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T22:53:42.887 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:53:42.897 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:53:42.951 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:42.952 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:42.953 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:42.951+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:42.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:42.967 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone14 from 0xe844af4 (overlap 0x392f6f) to 0x0 2024-09-14T22:53:43.018 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T22:53:43.035 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:43.033+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:43.070 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:43.070 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:43.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:43.136 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:43.135+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:43.184 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:43.184 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:43.193 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:43.212 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:43.212 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:43.212 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 55, 2024-09-14T22:53:43.212 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:53:43.212 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:53:43.212 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:53:42.893705+0000", 2024-09-14T22:53:43.212 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5801984, 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "noscrub", 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:53:43.213 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:53:43.214 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:53:43.214 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:53:43.214 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:53:43.214 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:53:43.214 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:53:43.214 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:53:43.215 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:53:43.215 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:43.216 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:43.217 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:53:43.218 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:53:43.219 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:43.219 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:43.219 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:43.219 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:43.219 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:43.219 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:43.219 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:43.219 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:43.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:43.221 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:53:43.222 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.94651603698730469, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.94999998807907104, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.94999998807907104 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:43.223 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:53:43.224 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:43.225 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:43.226 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:43.227 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:43.228 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 3, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 3, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.75, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2.25, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2.25, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.94651603698730469, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.94999998807907104, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.94999998807907104 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:53:43.229 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:53:43.230 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:53:43.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "55", 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 17, 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 55, 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:53:43.232 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:53:43.233 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:53:43.234 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.75, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.75, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.75, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.75, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.94651603698730469, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.94999998807907104, 2024-09-14T22:53:43.235 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.94999998807907104 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.5721282958984375, 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:43.236 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:43.237 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.238 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.239 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.240 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:53:43.241 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 47, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:53:43.242 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:43.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:43.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:43.246 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:53:43.247 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.248 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:43.249 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:43.250 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:53:43.251 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:53:43.252 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0.5237274169921875, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.253 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.254 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:43.255 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:43.256 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.257 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:43.258 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 50, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:43.264 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.266 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.267 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:53:43.268 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:43.269 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:53:43.270 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:53:43.271 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:53:43.272 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:53:43.273 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.274 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 50, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:53:43.275 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.276 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:43.277 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:53:43.278 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:53:43.279 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:43.280 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:43.281 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:53:43.282 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:43.283 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:53:43.284 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:53:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "2.6", 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: 5, 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [], 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [], 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:53:43.287 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [], 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:53:43.288 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:53:43.289 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:53:43.289 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:53:43.289 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:53:43.289 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:53:43.289 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:53:43.289 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:53:43.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:43.289 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:43.291 INFO:tasks.thrashosds.thrasher:j is {'epoch': 55, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:53:42.893705+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9465160369873047, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.949999988079071}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9465160369873047, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.949999988079071}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 55, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9465160369873047, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.949999988079071}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5721282958984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 0.5237274169921875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [5, 2]}, {'pgid': '3.0', 'osds': [6, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:53:43.291 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:53:43.292 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:43.289+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:43.308 INFO:teuthology.orchestra.run.smithi076.stdout:1214 write 0x848b89 thru 0x856bf4 (0xe06c bytes) 2024-09-14T22:53:43.337 INFO:teuthology.orchestra.run.smithi076.stdout:1215 writesame 0xec9bf1d thru 0xeca38bb data_size 0xd(0x799f bytes) 2024-09-14T22:53:43.339 INFO:teuthology.orchestra.run.smithi076.stdout:1216 write 0x376c4ae thru 0x376d1d8 (0xd2b bytes) 2024-09-14T22:53:43.343 INFO:teuthology.orchestra.run.smithi076.stdout:1217 writesame 0x3c569c7 thru 0x3c58eb1 data_size 0x1(0x24eb bytes) 2024-09-14T22:53:43.349 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.349 INFO:teuthology.orchestra.run.smithi076.stdout:1220 writesame 0xe9424ea thru 0xe94483e data_size 0x2d(0x2355 bytes) 2024-09-14T22:53:43.367 INFO:teuthology.orchestra.run.smithi076.stdout:1222 trunc from 0xeca38bc to 0x30c3d2 2024-09-14T22:53:43.378 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.382 INFO:teuthology.orchestra.run.smithi076.stdout:1224 trunc from 0x30c3d2 to 0x3a7fa8d 2024-09-14T22:53:43.387 INFO:teuthology.orchestra.run.smithi076.stdout:1225 read 0xc21a30 thru 0xc288bb (0x6e8c bytes) 2024-09-14T22:53:43.391 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:43.389+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:43.393 INFO:teuthology.orchestra.run.smithi076.stdout:1227 write 0x3c4a30c thru 0x3c51647 (0x733c bytes) 2024-09-14T22:53:43.395 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.395 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.399 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:43.399 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:43.408 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:43.410 INFO:teuthology.orchestra.run.smithi076.stdout:1231 write 0xbde4fad thru 0xbdec9f7 (0x7a4b bytes) 2024-09-14T22:53:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:1232 read 0x3724c5a thru 0x37294f2 (0x4899 bytes) 2024-09-14T22:53:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:1238 read 0xab10ee5 thru 0xab11049 (0x165 bytes) 2024-09-14T22:53:43.422 INFO:teuthology.orchestra.run.smithi076.stdout:1239 write 0xe552434 thru 0xe55ae5e (0x8a2b bytes) 2024-09-14T22:53:43.423 INFO:teuthology.orchestra.run.smithi076.stdout:1241 write 0x250b54f thru 0x250b75f (0x211 bytes) 2024-09-14T22:53:43.427 INFO:teuthology.orchestra.run.smithi076.stdout:1243 punch from 0xb82ece5 to 0xb838b0d, (0x9e28 bytes) 2024-09-14T22:53:43.427 INFO:teuthology.orchestra.run.smithi076.stdout:1244 punch from 0x761359b to 0x761ee00, (0xb865 bytes) 2024-09-14T22:53:43.427 INFO:teuthology.orchestra.run.smithi076.stdout:1245 writesame 0x7b35248 thru 0x7b3a654 data_size 0x1(0x540d bytes) 2024-09-14T22:53:43.442 INFO:teuthology.orchestra.run.smithi076.stdout:1247 read 0x8d984ce thru 0x8da079b (0x82ce bytes) 2024-09-14T22:53:43.443 INFO:teuthology.orchestra.run.smithi076.stdout:1248 write 0xd6d630c thru 0xd6d7037 (0xd2c bytes) 2024-09-14T22:53:43.447 INFO:teuthology.orchestra.run.smithi076.stdout:1249 writesame 0xc8e422 thru 0xc92415 data_size 0x2(0x3ff4 bytes) 2024-09-14T22:53:43.455 INFO:teuthology.orchestra.run.smithi076.stdout:1250 writesame 0xc7cbd35 thru 0xc7d3a6b data_size 0x5(0x7d37 bytes) 2024-09-14T22:53:43.462 INFO:teuthology.orchestra.run.smithi076.stdout:1251 write 0x4332aa2 thru 0x433e78e (0xbced bytes) 2024-09-14T22:53:43.476 INFO:teuthology.orchestra.run.smithi076.stdout:1252 trunc from 0xe55ae5f to 0x5a4b0a4 2024-09-14T22:53:43.492 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:43.491+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:43.496 INFO:teuthology.orchestra.run.smithi076.stdout:skipping zero size read 2024-09-14T22:53:43.496 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.499 INFO:teuthology.orchestra.run.smithi076.stdout:1255 trunc from 0x5a4b0a4 to 0x36238ca 2024-09-14T22:53:43.509 INFO:teuthology.orchestra.run.smithi076.stdout:1256 read 0x32ace76 thru 0x32b2c7e (0x5e09 bytes) 2024-09-14T22:53:43.518 INFO:teuthology.orchestra.run.smithi076.stdout:1257 trunc from 0x36238ca to 0xaf02673 2024-09-14T22:53:43.523 INFO:teuthology.orchestra.run.smithi076.stdout:1258 writesame 0x9cd1a89 thru 0x9cdb679 data_size 0x7(0x9bf1 bytes) 2024-09-14T22:53:43.530 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:43.530 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:43.540 INFO:teuthology.orchestra.run.smithi076.stdout:1259 writesame 0xec07be5 thru 0xec0c0e4 data_size 0x18(0x4500 bytes) 2024-09-14T22:53:43.542 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:43.542 INFO:teuthology.orchestra.run.smithi076.stdout:1260 writesame 0x88d20b3 thru 0x88dc83f data_size 0x1(0xa78d bytes) 2024-09-14T22:53:43.570 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.570 INFO:teuthology.orchestra.run.smithi076.stdout:1262 write 0x39192c7 thru 0x39215b8 (0x82f2 bytes) 2024-09-14T22:53:43.584 INFO:teuthology.orchestra.run.smithi076.stdout:1263 trunc from 0xec0c0e5 to 0x3c83dd5 2024-09-14T22:53:43.593 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:43.592+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:43.602 INFO:teuthology.orchestra.run.smithi076.stdout:1264 write 0xf4a122 thru 0xf5230e (0x81ed bytes) 2024-09-14T22:53:43.604 INFO:teuthology.orchestra.run.smithi076.stdout:1265 punch from 0x1de4e7e to 0x1de9f4e, (0x50d0 bytes) 2024-09-14T22:53:43.604 INFO:teuthology.orchestra.run.smithi076.stdout:1266 punch from 0xce3728 to 0xceddff, (0xa6d7 bytes) 2024-09-14T22:53:43.604 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.604 INFO:teuthology.orchestra.run.smithi076.stdout:1268 writesame 0x25a8b72 thru 0x25b2236 data_size 0x1(0x96c5 bytes) 2024-09-14T22:53:43.624 INFO:teuthology.orchestra.run.smithi076.stdout:1269 punch from 0x362425b to 0x363102c, (0xcdd1 bytes) 2024-09-14T22:53:43.635 INFO:teuthology.orchestra.run.smithi076.stdout:1271 write 0x757577d thru 0x75809e1 (0xb265 bytes) 2024-09-14T22:53:43.645 INFO:teuthology.orchestra.run.smithi076.stdout:1272 write 0x9055562 thru 0x905d475 (0x7f14 bytes) 2024-09-14T22:53:43.648 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:53:43.648 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:53:43.649 INFO:teuthology.orchestra.run.smithi076.stdout:1275 read 0x74908d1 thru 0x749e2ec (0xda1c bytes) 2024-09-14T22:53:43.650 INFO:teuthology.orchestra.run.smithi076.stdout:1276 write 0x3426efe thru 0x342d98d (0x6a90 bytes) 2024-09-14T22:53:43.657 INFO:teuthology.orchestra.run.smithi076.stdout:1278 writesame 0xa1fa4ca thru 0xa207ea5 data_size 0x49(0xd9dc bytes) 2024-09-14T22:53:43.657 INFO:teuthology.orchestra.run.smithi076.stdout:1280 write 0x6f117a2 thru 0x6f1f065 (0xd8c4 bytes) 2024-09-14T22:53:43.662 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-14T22:53:42.901972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419255016,"num_objects":1438,"num_object_clones":173,"num_object_copies":2876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1438,"num_whiteouts":1081,"num_read":141029,"num_read_kb":427868,"num_write":3958,"num_write_kb":249623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":11304792,"num_keys_recovered":9,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4380,"ondisk_log_size":4380,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":622012,"kb_used_data":364108,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754352708,"statfs":{"total":773094113280,"available":772457172992,"internally_reserved":0,"allocated":372846592,"data_stored":456514171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,8,40,60,58],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":66233813,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15573,"num_read_kb":40488,"num_write":136,"num_write_kb":49619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025494"},"pg_stats":[{"pgid":"3.1f","version":"43'208","reported_seq":6451,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.878790+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:53:38.878790+0000","last_peered":"2024-09-14T22:53:38.878790+0000","last_clean":"2024-09-14T22:53:38.878790+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:53:38.878790+0000","last_undegraded":"2024-09-14T22:53:38.878790+0000","last_fullsized":"2024-09-14T22:53:38.878790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:33.763829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":3540374,"num_objects":45,"num_object_clones":6,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":32,"num_read":7381,"num_read_kb":8051,"num_write":259,"num_write_kb":1795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"51'138","reported_seq":6914,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.156707+0000","last_change":"2024-09-14T22:53:32.831259+0000","last_active":"2024-09-14T22:53:39.156707+0000","last_peered":"2024-09-14T22:53:39.156707+0000","last_clean":"2024-09-14T22:53:39.156707+0000","last_became_active":"2024-09-14T22:53:32.751884+0000","last_became_peered":"2024-09-14T22:53:32.751884+0000","last_unstale":"2024-09-14T22:53:39.156707+0000","last_undegraded":"2024-09-14T22:53:39.156707+0000","last_fullsized":"2024-09-14T22:53:39.156707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:40:32.325922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098261599999999996,"stat_sum":{"num_bytes":7835229,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":33,"num_read":6852,"num_read_kb":9311,"num_write":132,"num_write_kb":3401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":301575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"50'99","reported_seq":6889,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880257+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:53:38.880257+0000","last_peered":"2024-09-14T22:53:38.880257+0000","last_clean":"2024-09-14T22:53:38.880257+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:53:38.880257+0000","last_undegraded":"2024-09-14T22:53:38.880257+0000","last_fullsized":"2024-09-14T22:53:38.880257+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:52:37.976057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":24179071,"num_objects":51,"num_object_clones":6,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":37,"num_read":6771,"num_read_kb":32389,"num_write":91,"num_write_kb":28077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"44'197","reported_seq":730,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880125+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:53:38.880125+0000","last_peered":"2024-09-14T22:53:38.880125+0000","last_clean":"2024-09-14T22:53:38.880125+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:53:38.880125+0000","last_undegraded":"2024-09-14T22:53:38.880125+0000","last_fullsized":"2024-09-14T22:53:38.880125+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:03.174348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":6328249,"num_objects":38,"num_object_clones":4,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":30,"num_read":1058,"num_read_kb":1362,"num_write":246,"num_write_kb":1129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"43'87","reported_seq":562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.878672+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:53:38.878672+0000","last_peered":"2024-09-14T22:53:38.878672+0000","last_clean":"2024-09-14T22:53:38.878672+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:53:38.878672+0000","last_undegraded":"2024-09-14T22:53:38.878672+0000","last_fullsized":"2024-09-14T22:53:38.878672+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:13:24.113228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":11403306,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":25,"num_read":503,"num_read_kb":19685,"num_write":82,"num_write_kb":9443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"46'148","reported_seq":9513,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.158579+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:53:39.158579+0000","last_peered":"2024-09-14T22:53:39.158579+0000","last_clean":"2024-09-14T22:53:39.158579+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:53:39.158579+0000","last_undegraded":"2024-09-14T22:53:39.158579+0000","last_fullsized":"2024-09-14T22:53:39.158579+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:07.873736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":15947382,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":35,"num_read":9592,"num_read_kb":24991,"num_write":113,"num_write_kb":7773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"51'135","reported_seq":10360,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.115713+0000","last_change":"2024-09-14T22:53:08.766652+0000","last_active":"2024-09-14T22:53:39.115713+0000","last_peered":"2024-09-14T22:53:39.115713+0000","last_clean":"2024-09-14T22:53:39.115713+0000","last_became_active":"2024-09-14T22:53:06.760167+0000","last_became_peered":"2024-09-14T22:53:06.760167+0000","last_unstale":"2024-09-14T22:53:39.115713+0000","last_undegraded":"2024-09-14T22:53:39.115713+0000","last_fullsized":"2024-09-14T22:53:39.115713+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:35:50.617380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":25534638,"num_objects":56,"num_object_clones":8,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":44,"num_read":10439,"num_read_kb":15676,"num_write":87,"num_write_kb":17559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"43'97","reported_seq":570,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.878464+0000","last_change":"2024-09-14T22:53:32.823044+0000","last_active":"2024-09-14T22:53:38.878464+0000","last_peered":"2024-09-14T22:53:38.878464+0000","last_clean":"2024-09-14T22:53:38.878464+0000","last_became_active":"2024-09-14T22:53:32.749192+0000","last_became_peered":"2024-09-14T22:53:32.749192+0000","last_unstale":"2024-09-14T22:53:38.878464+0000","last_undegraded":"2024-09-14T22:53:38.878464+0000","last_fullsized":"2024-09-14T22:53:38.878464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:59:54.359660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062866099999999998,"stat_sum":{"num_bytes":2062336,"num_objects":43,"num_object_clones":2,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":38,"num_read":340,"num_read_kb":3166,"num_write":63,"num_write_kb":1206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1050624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"46'105","reported_seq":637,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.882693+0000","last_change":"2024-09-14T22:53:38.882316+0000","last_active":"2024-09-14T22:53:38.882693+0000","last_peered":"2024-09-14T22:53:38.882693+0000","last_clean":"2024-09-14T22:53:38.882693+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:53:38.882693+0000","last_undegraded":"2024-09-14T22:53:38.882693+0000","last_fullsized":"2024-09-14T22:53:38.882693+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:35:39.952006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":2097234,"num_objects":34,"num_object_clones":1,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":26,"num_read":865,"num_read_kb":4865,"num_write":87,"num_write_kb":5159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"51'129","reported_seq":7444,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.999671+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:53:38.999671+0000","last_peered":"2024-09-14T22:53:38.999671+0000","last_clean":"2024-09-14T22:53:38.999671+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:53:38.999671+0000","last_undegraded":"2024-09-14T22:53:38.999671+0000","last_fullsized":"2024-09-14T22:53:38.999671+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:05:04.631044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":41379194,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":33,"num_read":8102,"num_read_kb":28324,"num_write":90,"num_write_kb":10396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"51'102","reported_seq":7533,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.945107+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:53:38.945107+0000","last_peered":"2024-09-14T22:53:38.945107+0000","last_clean":"2024-09-14T22:53:38.945107+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:53:38.945107+0000","last_undegraded":"2024-09-14T22:53:38.945107+0000","last_fullsized":"2024-09-14T22:53:38.945107+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:56:29.566217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":16807480,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":31,"num_read":7919,"num_read_kb":26319,"num_write":60,"num_write_kb":656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"43'123","reported_seq":6905,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.877678+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:53:38.877678+0000","last_peered":"2024-09-14T22:53:38.877678+0000","last_clean":"2024-09-14T22:53:38.877678+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:53:38.877678+0000","last_undegraded":"2024-09-14T22:53:38.877678+0000","last_fullsized":"2024-09-14T22:53:38.877678+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:06:27.450028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":29256911,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":31,"num_read":6808,"num_read_kb":36162,"num_write":109,"num_write_kb":14326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.888746+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:53:38.888746+0000","last_peered":"2024-09-14T22:53:38.888746+0000","last_clean":"2024-09-14T22:53:38.888746+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:53:38.888746+0000","last_undegraded":"2024-09-14T22:53:38.888746+0000","last_fullsized":"2024-09-14T22:53:38.888746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"51'278","reported_seq":5352,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.981217+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:53:38.981217+0000","last_peered":"2024-09-14T22:53:38.981217+0000","last_clean":"2024-09-14T22:53:38.981217+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:53:38.981217+0000","last_undegraded":"2024-09-14T22:53:38.981217+0000","last_fullsized":"2024-09-14T22:53:38.981217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:36:39.772637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":28458095,"num_objects":40,"num_object_clones":5,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":27,"num_read":5984,"num_read_kb":16098,"num_write":369,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880356+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:53:38.880356+0000","last_peered":"2024-09-14T22:53:38.880356+0000","last_clean":"2024-09-14T22:53:38.880356+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:53:38.880356+0000","last_undegraded":"2024-09-14T22:53:38.880356+0000","last_fullsized":"2024-09-14T22:53:38.880356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"43'78","reported_seq":572,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.892199+0000","last_change":"2024-09-14T22:53:08.762000+0000","last_active":"2024-09-14T22:53:38.892199+0000","last_peered":"2024-09-14T22:53:38.892199+0000","last_clean":"2024-09-14T22:53:38.892199+0000","last_became_active":"2024-09-14T22:53:06.761953+0000","last_became_peered":"2024-09-14T22:53:06.761953+0000","last_unstale":"2024-09-14T22:53:38.892199+0000","last_undegraded":"2024-09-14T22:53:38.892199+0000","last_fullsized":"2024-09-14T22:53:38.892199+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:25:30.983250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":4807107,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":463,"num_read_kb":6960,"num_write":55,"num_write_kb":6609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880602+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:38.880602+0000","last_peered":"2024-09-14T22:53:38.880602+0000","last_clean":"2024-09-14T22:53:38.880602+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:38.880602+0000","last_undegraded":"2024-09-14T22:53:38.880602+0000","last_fullsized":"2024-09-14T22:53:38.880602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"43'131","reported_seq":12044,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.878375+0000","last_change":"2024-09-14T22:53:08.766672+0000","last_active":"2024-09-14T22:53:38.878375+0000","last_peered":"2024-09-14T22:53:38.878375+0000","last_clean":"2024-09-14T22:53:38.878375+0000","last_became_active":"2024-09-14T22:53:06.755437+0000","last_became_peered":"2024-09-14T22:53:06.755437+0000","last_unstale":"2024-09-14T22:53:38.878375+0000","last_undegraded":"2024-09-14T22:53:38.878375+0000","last_fullsized":"2024-09-14T22:53:38.878375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:51:21.552158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":21912531,"num_objects":48,"num_object_clones":5,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":34,"num_read":12339,"num_read_kb":33480,"num_write":102,"num_write_kb":15459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":77,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.878469+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:53:38.878469+0000","last_peered":"2024-09-14T22:53:38.878469+0000","last_clean":"2024-09-14T22:53:38.878469+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:53:38.878469+0000","last_undegraded":"2024-09-14T22:53:38.878469+0000","last_fullsized":"2024-09-14T22:53:38.878469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"46'118","reported_seq":12397,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880344+0000","last_change":"2024-09-14T22:53:35.869981+0000","last_active":"2024-09-14T22:53:38.880344+0000","last_peered":"2024-09-14T22:53:38.880344+0000","last_clean":"2024-09-14T22:53:38.880344+0000","last_became_active":"2024-09-14T22:53:35.868554+0000","last_became_peered":"2024-09-14T22:53:35.868554+0000","last_unstale":"2024-09-14T22:53:38.880344+0000","last_undegraded":"2024-09-14T22:53:38.880344+0000","last_fullsized":"2024-09-14T22:53:38.880344+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:37:23.229685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":19829835,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":42,"num_read":10786,"num_read_kb":24570,"num_write":73,"num_write_kb":3723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":3794613,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"50'194","reported_seq":4214,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.117499+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:53:39.117499+0000","last_peered":"2024-09-14T22:53:39.117499+0000","last_clean":"2024-09-14T22:53:39.117499+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:53:39.117499+0000","last_undegraded":"2024-09-14T22:53:39.117499+0000","last_fullsized":"2024-09-14T22:53:39.117499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:34:01.322946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":31369983,"num_objects":59,"num_object_clones":10,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":42,"num_read":4697,"num_read_kb":18642,"num_write":167,"num_write_kb":28010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880283+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:53:38.880283+0000","last_peered":"2024-09-14T22:53:38.880283+0000","last_clean":"2024-09-14T22:53:38.880283+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:53:38.880283+0000","last_undegraded":"2024-09-14T22:53:38.880283+0000","last_fullsized":"2024-09-14T22:53:38.880283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"44'133","reported_seq":678,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.888691+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:53:38.888691+0000","last_peered":"2024-09-14T22:53:38.888691+0000","last_clean":"2024-09-14T22:53:38.888691+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:53:38.888691+0000","last_undegraded":"2024-09-14T22:53:38.888691+0000","last_fullsized":"2024-09-14T22:53:38.888691+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:35:37.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":8195607,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":23,"num_read":793,"num_read_kb":7070,"num_write":150,"num_write_kb":10189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":82,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.890775+0000","last_change":"2024-09-14T22:53:32.745426+0000","last_active":"2024-09-14T22:53:38.890775+0000","last_peered":"2024-09-14T22:53:38.890775+0000","last_clean":"2024-09-14T22:53:38.890775+0000","last_became_active":"2024-09-14T22:53:32.745120+0000","last_became_peered":"2024-09-14T22:53:32.745120+0000","last_unstale":"2024-09-14T22:53:38.890775+0000","last_undegraded":"2024-09-14T22:53:38.890775+0000","last_fullsized":"2024-09-14T22:53:38.890775+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:06:49.621943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'103","reported_seq":5647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.878462+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:53:38.878462+0000","last_peered":"2024-09-14T22:53:38.878462+0000","last_clean":"2024-09-14T22:53:38.878462+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:53:38.878462+0000","last_undegraded":"2024-09-14T22:53:38.878462+0000","last_fullsized":"2024-09-14T22:53:38.878462+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:49:39.245919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":4575676,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":31,"num_read":5775,"num_read_kb":8235,"num_write":69,"num_write_kb":1135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880178+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:38.880178+0000","last_peered":"2024-09-14T22:53:38.880178+0000","last_clean":"2024-09-14T22:53:38.880178+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:38.880178+0000","last_undegraded":"2024-09-14T22:53:38.880178+0000","last_fullsized":"2024-09-14T22:53:38.880178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"51'103","reported_seq":4067,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.129511+0000","last_change":"2024-09-14T22:53:32.825251+0000","last_active":"2024-09-14T22:53:39.129511+0000","last_peered":"2024-09-14T22:53:39.129511+0000","last_clean":"2024-09-14T22:53:39.129511+0000","last_became_active":"2024-09-14T22:53:32.746422+0000","last_became_peered":"2024-09-14T22:53:32.746422+0000","last_unstale":"2024-09-14T22:53:39.129511+0000","last_undegraded":"2024-09-14T22:53:39.129511+0000","last_fullsized":"2024-09-14T22:53:39.129511+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:56:12.157382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":1463948,"num_objects":43,"num_object_clones":2,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":39,"num_read":3832,"num_read_kb":4220,"num_write":71,"num_write_kb":14263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":57590,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":78,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880124+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:53:38.880124+0000","last_peered":"2024-09-14T22:53:38.880124+0000","last_clean":"2024-09-14T22:53:38.880124+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:53:38.880124+0000","last_undegraded":"2024-09-14T22:53:38.880124+0000","last_fullsized":"2024-09-14T22:53:38.880124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880272+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:38.880272+0000","last_peered":"2024-09-14T22:53:38.880272+0000","last_clean":"2024-09-14T22:53:38.880272+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:38.880272+0000","last_undegraded":"2024-09-14T22:53:38.880272+0000","last_fullsized":"2024-09-14T22:53:38.880272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"43'216","reported_seq":3903,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.878210+0000","last_change":"2024-09-14T22:53:35.869208+0000","last_active":"2024-09-14T22:53:38.878210+0000","last_peered":"2024-09-14T22:53:38.878210+0000","last_clean":"2024-09-14T22:53:38.878210+0000","last_became_active":"2024-09-14T22:53:35.868719+0000","last_became_peered":"2024-09-14T22:53:35.868719+0000","last_unstale":"2024-09-14T22:53:38.878210+0000","last_undegraded":"2024-09-14T22:53:38.878210+0000","last_fullsized":"2024-09-14T22:53:38.878210+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:03:27.842147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":9778982,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":41,"num_read":4629,"num_read_kb":14734,"num_write":185,"num_write_kb":8760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":4433187,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"43'90","reported_seq":379,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.877540+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:53:38.877540+0000","last_peered":"2024-09-14T22:53:38.877540+0000","last_clean":"2024-09-14T22:53:38.877540+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:53:38.877540+0000","last_undegraded":"2024-09-14T22:53:38.877540+0000","last_fullsized":"2024-09-14T22:53:38.877540+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:48:09.396185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":1251395,"num_objects":46,"num_object_clones":6,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":39,"num_read":261,"num_read_kb":1171,"num_write":49,"num_write_kb":997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"51'90","reported_seq":276,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.114985+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:53:39.114985+0000","last_peered":"2024-09-14T22:53:39.114985+0000","last_clean":"2024-09-14T22:53:39.114985+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:53:39.114985+0000","last_undegraded":"2024-09-14T22:53:39.114985+0000","last_fullsized":"2024-09-14T22:53:39.114985+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:06:48.404005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":7094832,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":30,"num_read":156,"num_read_kb":1873,"num_write":61,"num_write_kb":1018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"51'121","reported_seq":809,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.172466+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:53:39.172466+0000","last_peered":"2024-09-14T22:53:39.172466+0000","last_clean":"2024-09-14T22:53:39.172466+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:53:39.172466+0000","last_undegraded":"2024-09-14T22:53:39.172466+0000","last_fullsized":"2024-09-14T22:53:39.172466+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:44:29.795146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":4341776,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":37,"num_read":901,"num_read_kb":8083,"num_write":86,"num_write_kb":3574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"51'120","reported_seq":3554,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.151459+0000","last_change":"2024-09-14T22:53:35.869109+0000","last_active":"2024-09-14T22:53:39.151459+0000","last_peered":"2024-09-14T22:53:39.151459+0000","last_clean":"2024-09-14T22:53:39.151459+0000","last_became_active":"2024-09-14T22:53:35.868711+0000","last_became_peered":"2024-09-14T22:53:35.868711+0000","last_unstale":"2024-09-14T22:53:39.151459+0000","last_undegraded":"2024-09-14T22:53:39.151459+0000","last_fullsized":"2024-09-14T22:53:39.151459+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:20:19.726370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":11717466,"num_objects":52,"num_object_clones":4,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":37,"num_read":3411,"num_read_kb":9724,"num_write":99,"num_write_kb":1821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1667203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"43'110","reported_seq":2744,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880498+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:53:38.880498+0000","last_peered":"2024-09-14T22:53:38.880498+0000","last_clean":"2024-09-14T22:53:38.880498+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:53:38.880498+0000","last_undegraded":"2024-09-14T22:53:38.880498+0000","last_fullsized":"2024-09-14T22:53:38.880498+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:53:14.395206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":7140638,"num_objects":51,"num_object_clones":5,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":41,"num_read":2611,"num_read_kb":10937,"num_write":69,"num_write_kb":3284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"43'94","reported_seq":471,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880332+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:53:38.880332+0000","last_peered":"2024-09-14T22:53:38.880332+0000","last_clean":"2024-09-14T22:53:38.880332+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:53:38.880332+0000","last_undegraded":"2024-09-14T22:53:38.880332+0000","last_fullsized":"2024-09-14T22:53:38.880332+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:14:42.667117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":11375296,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":32,"num_read":344,"num_read_kb":3463,"num_write":62,"num_write_kb":2024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"43'105","reported_seq":3307,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880496+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:53:38.880496+0000","last_peered":"2024-09-14T22:53:38.880496+0000","last_clean":"2024-09-14T22:53:38.880496+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:53:38.880496+0000","last_undegraded":"2024-09-14T22:53:38.880496+0000","last_fullsized":"2024-09-14T22:53:38.880496+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:17:15.476137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":6182249,"num_objects":41,"num_object_clones":5,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":30,"num_read":3250,"num_read_kb":10891,"num_write":84,"num_write_kb":4582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"44'113","reported_seq":7418,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.888695+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:53:38.888695+0000","last_peered":"2024-09-14T22:53:38.888695+0000","last_clean":"2024-09-14T22:53:38.888695+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:53:38.888695+0000","last_undegraded":"2024-09-14T22:53:38.888695+0000","last_fullsized":"2024-09-14T22:53:38.888695+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:38:14.513233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":18196684,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":35,"num_read":7725,"num_read_kb":18359,"num_write":71,"num_write_kb":12882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"50'177","reported_seq":3683,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880572+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:53:38.880572+0000","last_peered":"2024-09-14T22:53:38.880572+0000","last_clean":"2024-09-14T22:53:38.880572+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:53:38.880572+0000","last_undegraded":"2024-09-14T22:53:38.880572+0000","last_fullsized":"2024-09-14T22:53:38.880572+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:59:07.340293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":12597330,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":29,"num_read":3965,"num_read_kb":14021,"num_write":232,"num_write_kb":12919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"51'201","reported_seq":438,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:39.163180+0000","last_change":"2024-09-14T22:53:38.883182+0000","last_active":"2024-09-14T22:53:39.163180+0000","last_peered":"2024-09-14T22:53:39.163180+0000","last_clean":"2024-09-14T22:53:39.163180+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:53:39.163180+0000","last_undegraded":"2024-09-14T22:53:39.163180+0000","last_fullsized":"2024-09-14T22:53:39.163180+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:44:21.308635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":19293182,"num_objects":43,"num_object_clones":4,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":34,"num_read":712,"num_read_kb":3233,"num_write":249,"num_write_kb":5316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"44'117","reported_seq":1791,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-14T22:53:38.880487+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:53:38.880487+0000","last_peered":"2024-09-14T22:53:38.880487+0000","last_clean":"2024-09-14T22:53:38.880487+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:53:38.880487+0000","last_undegraded":"2024-09-14T22:53:38.880487+0000","last_fullsized":"2024-09-14T22:53:38.880487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:48:32.972286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":2710613,"num_objects":47,"num_object_clones":3,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":40,"num_read":1649,"num_read_kb":1703,"num_write":75,"num_write_kb":903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":418664629,"num_objects":1434,"num_object_clones":173,"num_object_copies":2868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1434,"num_whiteouts":1081,"num_read":140913,"num_read_kb":427768,"num_write":3797,"num_write_kb":247205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":11304792,"num_keys_recovered":9,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":366944256,"data_stored":453271380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4260,"ondisk_log_size":4260,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83872,"kb_used_data":52680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94287968,"statfs":{"total":96636764160,"available":96550879232,"internally_reserved":0,"allocated":53944320,"data_stored":67816262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51036,"kb_used_data":19844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320804,"statfs":{"total":96636764160,"available":96584503296,"internally_reserved":0,"allocated":20320256,"data_stored":24526604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63800,"kb_used_data":32608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308040,"statfs":{"total":96636764160,"available":96571432960,"internally_reserved":0,"allocated":33390592,"data_stored":39207100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123948,"kb_used_data":88576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94247892,"statfs":{"total":96636764160,"available":96509841408,"internally_reserved":0,"allocated":90701824,"data_stored":109771575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,40,60,58],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56060,"kb_used_data":24868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315780,"statfs":{"total":96636764160,"available":96579358720,"internally_reserved":0,"allocated":25464832,"data_stored":26313720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52156,"kb_used_data":20972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94319684,"statfs":{"total":96636764160,"available":96583356416,"internally_reserved":0,"allocated":21475328,"data_stored":31043824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85928,"kb_used_data":54720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94285912,"statfs":{"total":96636764160,"available":96548773888,"internally_reserved":0,"allocated":56033280,"data_stored":77413948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105212,"kb_used_data":69840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94266628,"statfs":{"total":96636764160,"available":96529027072,"internally_reserved":0,"allocated":71516160,"data_stored":80421138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":70340608,"data_stored":79579693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":55447552,"data_stored":77162852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20893696,"data_stored":30792746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24817664,"data_stored":26003363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90120192,"data_stored":109520498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32804864,"data_stored":38956004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":19750912,"data_stored":24281407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":52768768,"data_stored":66974817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:53:43.664 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [3, 6] 2024-09-14T22:53:43.664 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.8', '3', '6'] 2024-09-14T22:53:43.664 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.8 3 6 2024-09-14T22:53:43.670 INFO:teuthology.orchestra.run.smithi076.stdout:1283 trunc from 0xa207ea6 to 0xe9ee64e 2024-09-14T22:53:43.675 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.675 INFO:teuthology.orchestra.run.smithi076.stdout:1285 writesame 0x3961b5b thru 0x3961dd9 data_size 0x3(0x27f bytes) 2024-09-14T22:53:43.677 INFO:teuthology.orchestra.run.smithi076.stdout:1286 read 0xd423c59 thru 0xd433427 (0xf7cf bytes) 2024-09-14T22:53:43.677 INFO:teuthology.orchestra.run.smithi076.stdout:1287 write 0xd77dd5a thru 0xd7853a8 (0x764f bytes) 2024-09-14T22:53:43.680 INFO:teuthology.orchestra.run.smithi076.stdout:1288 read 0x22bf4b5 thru 0x22c40f2 (0x4c3e bytes) 2024-09-14T22:53:43.681 INFO:teuthology.orchestra.run.smithi076.stdout:1289 writesame 0x1c74aa8 thru 0x1c7cbd0 data_size 0x11(0x8129 bytes) 2024-09-14T22:53:43.685 INFO:teuthology.orchestra.run.smithi076.stdout:1290 punch from 0x661d305 to 0x662a87b, (0xd576 bytes) 2024-09-14T22:53:43.685 INFO:teuthology.orchestra.run.smithi076.stdout:1291 write 0xb7017c9 thru 0xb70f4ef (0xdd27 bytes) 2024-09-14T22:53:43.693 INFO:teuthology.orchestra.run.smithi076.stdout:1293 trunc from 0xe9ee64e to 0xb199ab9 2024-09-14T22:53:43.694 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:43.692+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:43.701 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:43.701 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:43.703 INFO:teuthology.orchestra.run.smithi076.stdout:1294 read 0x17674db thru 0x176aa90 (0x35b6 bytes) 2024-09-14T22:53:43.703 INFO:teuthology.orchestra.run.smithi076.stdout:1295 write 0x5129bc5 thru 0x5134421 (0xa85d bytes) 2024-09-14T22:53:43.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:43.715 INFO:teuthology.orchestra.run.smithi076.stdout:1297 write 0xd748632 thru 0xd7567dc (0xe1ab bytes) 2024-09-14T22:53:43.718 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.722 INFO:teuthology.orchestra.run.smithi076.stdout:1300 trunc from 0xd7567dd to 0xadb95d2 2024-09-14T22:53:43.737 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.737 INFO:teuthology.orchestra.run.smithi076.stdout:1303 read 0x5d2dd2b thru 0x5d378dc (0x9bb2 bytes) 2024-09-14T22:53:43.737 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.737 INFO:teuthology.orchestra.run.smithi076.stdout:1306 read 0x448e568 thru 0x4499dcb (0xb864 bytes) 2024-09-14T22:53:43.738 INFO:teuthology.orchestra.run.smithi076.stdout:1307 read 0x8ccfe60 thru 0x8cd34a8 (0x3649 bytes) 2024-09-14T22:53:43.744 INFO:teuthology.orchestra.run.smithi076.stdout:1308 write 0xc8797c2 thru 0xc88240e (0x8c4d bytes) 2024-09-14T22:53:43.748 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.748 INFO:teuthology.orchestra.run.smithi076.stdout:1311 write 0x5029682 thru 0x503059a (0x6f19 bytes) 2024-09-14T22:53:43.749 INFO:teuthology.orchestra.run.smithi076.stdout:1312 read 0xa6fef1 thru 0xa7bda1 (0xbeb1 bytes) 2024-09-14T22:53:43.755 INFO:teuthology.orchestra.run.smithi076.stdout:1313 trunc from 0xc88240f to 0x86450f4 2024-09-14T22:53:43.766 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.774 INFO:teuthology.orchestra.run.smithi076.stdout:1315 writesame 0xa01c4de thru 0xa026796 data_size 0xb(0xa2b9 bytes) 2024-09-14T22:53:43.774 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.774 INFO:teuthology.orchestra.run.smithi076.stdout:1318 read 0x950d0d8 thru 0x9519d8a (0xccb3 bytes) 2024-09-14T22:53:43.774 INFO:teuthology.orchestra.run.smithi076.stdout:1319 read 0x740522 thru 0x748e9d (0x897c bytes) 2024-09-14T22:53:43.775 INFO:teuthology.orchestra.run.smithi076.stdout:1320 read 0x47c74e2 thru 0x47c7f98 (0xab7 bytes) 2024-09-14T22:53:43.775 INFO:teuthology.orchestra.run.smithi076.stdout:1321 write 0x537d4b4 thru 0x5387d8a (0xa8d7 bytes) 2024-09-14T22:53:43.776 INFO:teuthology.orchestra.run.smithi076.stdout:1322 punch from 0x84beb91 to 0x84cae7e, (0xc2ed bytes) 2024-09-14T22:53:43.783 INFO:teuthology.orchestra.run.smithi076.stdout:1323 write 0xa8a1d17 thru 0xa8a4775 (0x2a5f bytes) 2024-09-14T22:53:43.785 INFO:teuthology.orchestra.run.smithi076.stdout:1324 read 0x281098f thru 0x2812595 (0x1c07 bytes) 2024-09-14T22:53:43.795 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:43.792+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:43.795 INFO:teuthology.orchestra.run.smithi076.stdout:1326 write 0xcc9d600 thru 0xccac31c (0xed1d bytes) 2024-09-14T22:53:43.798 INFO:teuthology.orchestra.run.smithi076.stdout:1327 punch from 0xa43699c to 0xa43ea5a, (0x80be bytes) 2024-09-14T22:53:43.798 INFO:teuthology.orchestra.run.smithi076.stdout:1328 read 0x30ae821 thru 0x30b867a (0x9e5a bytes) 2024-09-14T22:53:43.799 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.799 INFO:teuthology.orchestra.run.smithi076.stdout:1330 writesame 0xb2a7a23 thru 0xb2addb0 data_size 0x1(0x638e bytes) 2024-09-14T22:53:43.815 INFO:teuthology.orchestra.run.smithi076.stdout:1331 read 0x5558608 thru 0x55595e5 (0xfde bytes) 2024-09-14T22:53:43.817 INFO:teuthology.orchestra.run.smithi076.stdout:1332 write 0x1ce712a thru 0x1cf06c9 (0x95a0 bytes) 2024-09-14T22:53:43.819 INFO:teuthology.orchestra.run.smithi076.stdout:1334 read 0x9fc492f thru 0x9fd3e9c (0xf56e bytes) 2024-09-14T22:53:43.819 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.819 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.819 INFO:teuthology.orchestra.run.smithi076.stdout:1339 read 0x7ffdfdb thru 0x8003678 (0x569e bytes) 2024-09-14T22:53:43.819 INFO:teuthology.orchestra.run.smithi076.stdout:1340 read 0x90c07d5 thru 0x90cb8c8 (0xb0f4 bytes) 2024-09-14T22:53:43.824 INFO:teuthology.orchestra.run.smithi076.stdout:1342 trunc from 0xccac31d to 0x9d7cf31 2024-09-14T22:53:43.837 INFO:teuthology.orchestra.run.smithi076.stdout:1343 read 0x5e0a89f thru 0x5e1960f (0xed71 bytes) 2024-09-14T22:53:43.837 INFO:teuthology.orchestra.run.smithi076.stdout:1345 writesame 0x7174421 thru 0x7181c3b data_size 0x1b(0xd81b bytes) 2024-09-14T22:53:43.842 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:43.842 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:43.844 INFO:teuthology.orchestra.run.smithi076.stdout:1347 trunc from 0x9d7cf31 to 0xcb50065 2024-09-14T22:53:43.854 INFO:teuthology.orchestra.run.smithi076.stdout:1349 trunc from 0xcb50065 to 0x98526dd 2024-09-14T22:53:43.856 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:43.867 INFO:teuthology.orchestra.run.smithi076.stdout:1350 writesame 0xbd1c46b thru 0xbd27289 data_size 0x5(0xae1f bytes) 2024-09-14T22:53:43.867 INFO:teuthology.orchestra.run.smithi076.stdout:1351 write 0x4686d2e thru 0x4692d43 (0xc016 bytes) 2024-09-14T22:53:43.870 INFO:teuthology.orchestra.run.smithi076.stdout:1352 read 0xaa0c4c1 thru 0xaa1520c (0x8d4c bytes) 2024-09-14T22:53:43.925 INFO:teuthology.orchestra.run.smithi076.stdout:1354 trunc from 0xbd2728a to 0x4491e0b 2024-09-14T22:53:43.925 INFO:teuthology.orchestra.run.smithi076.stdout:1355 write 0xcb42b2f thru 0xcb50207 (0xd6d9 bytes) 2024-09-14T22:53:43.925 INFO:teuthology.orchestra.run.smithi076.stdout:1357 write 0x48f276e thru 0x4900899 (0xe12c bytes) 2024-09-14T22:53:43.925 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.926 INFO:teuthology.orchestra.run.smithi076.stdout:1359 read 0x752361a thru 0x752864a (0x5031 bytes) 2024-09-14T22:53:43.926 INFO:teuthology.orchestra.run.smithi076.stdout:1360 punch from 0xbcd0ae9 to 0xbcd2078, (0x158f bytes) 2024-09-14T22:53:43.926 INFO:teuthology.orchestra.run.smithi076.stdout:1361 write 0xc0e2860 thru 0xc0f071d (0xdebe bytes) 2024-09-14T22:53:43.926 INFO:teuthology.orchestra.run.smithi076.stdout:1362 writesame 0x132d50f thru 0x133ca3a data_size 0x11(0xf52c bytes) 2024-09-14T22:53:43.926 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.926 INFO:teuthology.orchestra.run.smithi076.stdout:1364 write 0xa40343b thru 0xa40a986 (0x754c bytes) 2024-09-14T22:53:43.926 INFO:teuthology.orchestra.run.smithi076.stdout:1366 punch from 0x784f7b7 to 0x785f2a8, (0xfaf1 bytes) 2024-09-14T22:53:43.926 INFO:teuthology.orchestra.run.smithi076.stdout:1368 punch from 0x7734202 to 0x773abd8, (0x69d6 bytes) 2024-09-14T22:53:43.926 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:43.922+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:43.931 INFO:teuthology.orchestra.run.smithi076.stdout:1370 write 0xe15863b thru 0xe159528 (0xeee bytes) 2024-09-14T22:53:43.938 INFO:teuthology.orchestra.run.smithi076.stdout:1371 write 0xede3cce thru 0xede5ec0 (0x21f3 bytes) 2024-09-14T22:53:43.939 INFO:teuthology.orchestra.run.smithi076.stdout:1372 write 0xe2e2593 thru 0xe2ece6c (0xa8da bytes) 2024-09-14T22:53:43.941 INFO:teuthology.orchestra.run.smithi076.stdout:1373 write 0x712572c thru 0x712ae8e (0x5763 bytes) 2024-09-14T22:53:43.944 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.944 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.944 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.944 INFO:teuthology.orchestra.run.smithi076.stdout:1378 read 0x4873f03 thru 0x4883ac5 (0xfbc3 bytes) 2024-09-14T22:53:43.945 INFO:teuthology.orchestra.run.smithi076.stdout:1379 writesame 0x73e6406 thru 0x73eab4a data_size 0x29(0x4745 bytes) 2024-09-14T22:53:43.947 INFO:teuthology.orchestra.run.smithi076.stdout:1380 read 0x82b59b9 thru 0x82bc05d (0x66a5 bytes) 2024-09-14T22:53:43.947 INFO:teuthology.orchestra.run.smithi076.stdout:1381 write 0xa92eaeb thru 0xa93a015 (0xb52b bytes) 2024-09-14T22:53:43.948 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.948 INFO:teuthology.orchestra.run.smithi076.stdout:1383 read 0xdb4f293 thru 0xdb50806 (0x1574 bytes) 2024-09-14T22:53:43.948 INFO:teuthology.orchestra.run.smithi076.stdout:1384 writesame 0x38a17f0 thru 0x38adc07 data_size 0x64(0xc418 bytes) 2024-09-14T22:53:43.950 INFO:teuthology.orchestra.run.smithi076.stdout:1385 write 0xd8c0813 thru 0xd8c4b56 (0x4344 bytes) 2024-09-14T22:53:43.953 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.953 INFO:teuthology.orchestra.run.smithi076.stdout:1387 read 0x53fc50d thru 0x540526e (0x8d62 bytes) 2024-09-14T22:53:43.953 INFO:teuthology.orchestra.run.smithi076.stdout:1388 read 0x61aa383 thru 0x61b4d99 (0xaa17 bytes) 2024-09-14T22:53:43.958 INFO:teuthology.orchestra.run.smithi076.stdout:1389 trunc from 0xede5ec1 to 0xa840f9e 2024-09-14T22:53:43.975 INFO:teuthology.orchestra.run.smithi076.stdout:1390 punch from 0x923d255 to 0x9248cf0, (0xba9b bytes) 2024-09-14T22:53:43.975 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:43.975 INFO:teuthology.orchestra.run.smithi076.stdout:1393 read 0x51dc260 thru 0x51e201e (0x5dbf bytes) 2024-09-14T22:53:43.975 INFO:teuthology.orchestra.run.smithi076.stdout:1394 read 0x1b00de3 thru 0x1b0adc0 (0x9fde bytes) 2024-09-14T22:53:43.976 INFO:teuthology.orchestra.run.smithi076.stdout:1395 read 0x17bfc08 thru 0x17c6965 (0x6d5e bytes) 2024-09-14T22:53:43.977 INFO:teuthology.orchestra.run.smithi076.stdout:1399 trunc from 0xa840f9e to 0x907bbfb 2024-09-14T22:53:43.987 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:43.987 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:43.996 INFO:teuthology.orchestra.run.smithi076.stdout:1400 write 0xc845446 thru 0xc84bff9 (0x6bb4 bytes) 2024-09-14T22:53:43.999 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:44.000 INFO:teuthology.orchestra.run.smithi076.stdout:1402 punch from 0xd1f873 to 0xd2dbae, (0xe33b bytes) 2024-09-14T22:53:44.006 INFO:teuthology.orchestra.run.smithi076.stdout:1405 writesame 0xd8de58c thru 0xd8e1ab9 data_size 0x3(0x352e bytes) 2024-09-14T22:53:44.006 INFO:teuthology.orchestra.run.smithi076.stdout:1406 write 0xd580631 thru 0xd58b2c7 (0xac97 bytes) 2024-09-14T22:53:44.009 INFO:teuthology.orchestra.run.smithi076.stdout:1407 write 0x1273bd6 thru 0x1273efa (0x325 bytes) 2024-09-14T22:53:44.011 INFO:teuthology.orchestra.run.smithi076.stdout:1408 read 0x83ce6f3 thru 0x83cf54e (0xe5c bytes) 2024-09-14T22:53:44.011 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:44.011 INFO:teuthology.orchestra.run.smithi076.stdout:1410 trunc from 0xd8e1aba to 0xdd31324 2024-09-14T22:53:44.016 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:44.026 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:44.023+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:44.032 INFO:teuthology.orchestra.run.smithi076.stdout:1412 trunc from 0xdd31324 to 0x18be328 2024-09-14T22:53:44.056 INFO:teuthology.orchestra.run.smithi076.stdout:1413 trunc from 0x18be328 to 0x2f88986 2024-09-14T22:53:44.061 INFO:teuthology.orchestra.run.smithi076.stdout:1414 read 0x18b385 thru 0x199cc2 (0xe93e bytes) 2024-09-14T22:53:44.080 INFO:teuthology.orchestra.run.smithi076.stdout:1415 write 0xd5ad200 thru 0xd5ba4b8 (0xd2b9 bytes) 2024-09-14T22:53:44.083 INFO:teuthology.orchestra.run.smithi076.stdout:1417 read 0xaff78c5 thru 0xaffed5d (0x7499 bytes) 2024-09-14T22:53:44.083 INFO:teuthology.orchestra.run.smithi076.stdout:1418 punch from 0x6c14ae7 to 0x6c22134, (0xd64d bytes) 2024-09-14T22:53:44.083 INFO:teuthology.orchestra.run.smithi076.stdout:1420 write 0x8fc6315 thru 0x8fcead4 (0x87c0 bytes) 2024-09-14T22:53:44.087 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:44.087 INFO:teuthology.orchestra.run.smithi076.stdout:1422 read 0x394ce85 thru 0x395663d (0x97b9 bytes) 2024-09-14T22:53:44.087 INFO:teuthology.orchestra.run.smithi076.stdout:1423 punch from 0xa7cf833 to 0xa7da8a6, (0xb073 bytes) 2024-09-14T22:53:44.087 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:44.087 INFO:teuthology.orchestra.run.smithi076.stdout:1426 writesame 0x81de83c thru 0x81eabe1 data_size 0x4f(0xc3a6 bytes) 2024-09-14T22:53:44.088 INFO:teuthology.orchestra.run.smithi076.stdout:1428 punch from 0x594e11e to 0x59517da, (0x36bc bytes) 2024-09-14T22:53:44.088 INFO:teuthology.orchestra.run.smithi076.stdout:1429 read 0x31fd069 thru 0x32079b8 (0xa950 bytes) 2024-09-14T22:53:44.088 INFO:teuthology.orchestra.run.smithi076.stdout:1431 writesame 0x7dacf6b thru 0x7db806d data_size 0x5f(0xb103 bytes) 2024-09-14T22:53:44.091 INFO:teuthology.orchestra.run.smithi076.stdout:1432 read 0xaddf74f thru 0xadebd44 (0xc5f6 bytes) 2024-09-14T22:53:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:1433 read 0x182da54 thru 0x1834c97 (0x7244 bytes) 2024-09-14T22:53:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:1436 read 0x78e20ea thru 0x78f1991 (0xf8a8 bytes) 2024-09-14T22:53:44.092 INFO:teuthology.orchestra.run.smithi076.stdout:1437 read 0xcea8f59 thru 0xceb5d8c (0xce34 bytes) 2024-09-14T22:53:44.099 INFO:teuthology.orchestra.run.smithi076.stdout:1438 writesame 0xedbe040 thru 0xedc7e8a data_size 0x7(0x9e4b bytes) 2024-09-14T22:53:44.100 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:44.100 INFO:teuthology.orchestra.run.smithi076.stdout:1441 read 0x2870cfc thru 0x2873de8 (0x30ed bytes) 2024-09-14T22:53:44.100 INFO:teuthology.orchestra.run.smithi076.stdout:1442 read 0xeb26435 thru 0xeb27c51 (0x181d bytes) 2024-09-14T22:53:44.101 INFO:teuthology.orchestra.run.smithi076.stdout:1443 write 0xa1cb2fa thru 0xa1d7bb3 (0xc8ba bytes) 2024-09-14T22:53:44.103 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:44.104 INFO:teuthology.orchestra.run.smithi076.stdout:1445 punch from 0x52f0f0f to 0x5300f05, (0xfff6 bytes) 2024-09-14T22:53:44.104 INFO:teuthology.orchestra.run.smithi076.stdout:1446 write 0x2fe040c thru 0x2fe8362 (0x7f57 bytes) 2024-09-14T22:53:44.106 INFO:teuthology.orchestra.run.smithi076.stdout:1447 write 0x137adde thru 0x13896e4 (0xe907 bytes) 2024-09-14T22:53:44.108 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:44.108 INFO:teuthology.orchestra.run.smithi076.stdout:1450 punch from 0xe3a9a64 to 0xe3b52c0, (0xb85c bytes) 2024-09-14T22:53:44.108 INFO:teuthology.orchestra.run.smithi076.stdout:1451 read 0xae4b2c7 thru 0xae572d7 (0xc011 bytes) 2024-09-14T22:53:44.108 INFO:teuthology.orchestra.run.smithi076.stdout:1452 read 0xac2e422 thru 0xac3367d (0x525c bytes) 2024-09-14T22:53:44.108 INFO:teuthology.orchestra.run.smithi076.stdout:1453 writesame 0x58b598f thru 0x58bd60a data_size 0x3e(0x7c7c bytes) 2024-09-14T22:53:44.112 INFO:teuthology.orchestra.run.smithi076.stdout:1454 read 0x9c63a2d thru 0x9c67159 (0x372d bytes) 2024-09-14T22:53:44.112 INFO:teuthology.orchestra.run.smithi076.stdout:1455 writesame 0xac79a thru 0xad55c data_size 0x1(0xdc3 bytes) 2024-09-14T22:53:44.120 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:44.120 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:44.127 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:44.124+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:44.132 INFO:teuthology.orchestra.run.smithi076.stdout:1457 trunc from 0xedc7e8b to 0x97cb93 2024-09-14T22:53:44.134 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:44.150 INFO:teuthology.orchestra.run.smithi076.stdout:1458 read 0x2e49d thru 0x36718 (0x827c bytes) 2024-09-14T22:53:44.170 INFO:teuthology.orchestra.run.smithi076.stdout:1461 write 0xd1cbc32 thru 0xd1d6019 (0xa3e8 bytes) 2024-09-14T22:53:44.174 INFO:teuthology.orchestra.run.smithi076.stdout:1462 trunc from 0xd1d601a to 0xcb527d4 2024-09-14T22:53:44.183 INFO:teuthology.orchestra.run.smithi076.stdout:1463 clone 16 order 21 su 32768 sc 6 2024-09-14T22:53:44.227 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:44.226+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:44.257 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:44.258 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:44.266 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:44.328 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:44.327+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:44.410 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:44.410 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:44.413 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:44.418 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:44.429 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:44.428+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:44.529 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:44.527+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:44.536 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:44.537 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:44.554 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:44.554 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:44.566 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:44.566 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:44.570 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:44.575 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:44.587 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:44.588 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:44.605 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:44.606 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:44.623 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:44.623 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:44.630 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:44.628+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:44.639 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:44.639 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:44.656 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:44.656 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:44.694 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:44.694 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4232.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 47.161842270000001, 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010310849999999999, 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530813+0000", 2024-09-14T22:53:44.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8500000000000002e-07 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530823+0000", 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0277e-05 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530835+0000", 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1459e-05 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530845+0000", 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0239e-05 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530907+0000", 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1981000000000002e-05 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.530934+0000", 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6701999999999999e-05 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.531810+0000", 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00087597399999999996 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:52:57.531844+0000", 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4068e-05 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8992 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'23, mlcod=33'21)", 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 26.786280773000001, 2024-09-14T22:53:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0035453559999999999, 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906376+0000", 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.42e-06 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906968+0000", 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00059252399999999998 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906990+0000", 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1498999999999999e-05 2024-09-14T22:53:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906999+0000", 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2299999999999997e-06 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907058+0000", 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9505e-05 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907082+0000", 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3298999999999998e-05 2024-09-14T22:53:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909874+0000", 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027926689999999998 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909919+0000", 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5210000000000003e-05 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8998 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'25, mlcod=33'21)", 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 26.784612839000001, 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003268272, 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908043+0000", 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908054+0000", 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1654999999999999e-05 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908066+0000", 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1321000000000001e-05 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908077+0000", 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1781e-05 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908149+0000", 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1402000000000005e-05 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908170+0000", 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0746000000000001e-05 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911259+0000", 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030889839999999999 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911310+0000", 2024-09-14T22:53:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1684999999999997e-05 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "pg_update_log_missing(3.9 epoch 34/24 rep_tid 166 entries 34'26 (0'0) error 3:94c3b88b:::rbd_data.113535cec067.000000000000010d:head by client.4405.0:10843 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 34'23)", 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.114828+0000", 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 26.577827030000002, 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00099293600000000004, 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114828+0000", 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114828+0000", 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114829+0000", 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114843+0000", 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3687e-05 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114871+0000", 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8269e-05 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114888+0000", 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.719e-05 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.114980+0000", 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1689999999999998e-05 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.115821+0000", 2024-09-14T22:53:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00084082199999999997 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15153 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'58, mlcod=34'56)", 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 25.982438421000001, 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043627650000000002, 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710218+0000", 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.291e-06 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710957+0000", 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00073905599999999998 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710977+0000", 2024-09-14T22:53:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0506999999999999e-05 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710986+0000", 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0860000000000008e-06 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711048+0000", 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1130999999999995e-05 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711072+0000", 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4167000000000001e-05 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714533+0000", 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034611070000000002 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714579+0000", 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6419999999999999e-05 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15154 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'60, mlcod=34'56)", 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 25.981087666000001, 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00342618, 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711568+0000", 2024-09-14T22:53:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0900000000000002e-07 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711574+0000", 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5119999999999997e-06 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711583+0000", 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7579999999999998e-06 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711589+0000", 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9619999999999998e-06 2024-09-14T22:53:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711692+0000", 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000103237 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711719+0000", 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7124999999999999e-05 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714953+0000", 2024-09-14T22:53:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032341240000000001 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714993+0000", 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9953000000000002e-05 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77119 3.9 e40/24 3:9667b220:::rbd_data.113564f98e71.0000000000000024:head v 40'54, mlcod=39'51)", 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:26.993504+0000", 2024-09-14T22:53:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 17.699150462999999, 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010398370000000001, 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993504+0000", 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993504+0000", 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993505+0000", 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4700000000000002e-07 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993511+0000", 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2179999999999996e-06 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993529+0000", 2024-09-14T22:53:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8280000000000001e-05 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993539+0000", 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7820000000000005e-06 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993592+0000", 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3183999999999998e-05 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.993616+0000", 2024-09-14T22:53:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4043999999999999e-05 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.994500+0000", 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00088420199999999995 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:26.994544+0000", 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3779999999999998e-05 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77339 3.9 e40/24 3:939e5cf2:::rbd_data.113564f98e71.0000000000000044:head v 40'60, mlcod=40'58)", 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 17.660471899000001, 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0019342230000000001, 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032184+0000", 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032626+0000", 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000441369 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032644+0000", 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8029999999999998e-05 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032652+0000", 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7760000000000003e-06 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032720+0000", 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032743+0000", 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2823000000000002e-05 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034080+0000", 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001337705 2024-09-14T22:53:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034117+0000", 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6854000000000003e-05 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77340 3.9 e40/24 3:939e5cf2:::rbd_data.113564f98e71.0000000000000044:head v 40'62, mlcod=40'58)", 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 17.659582857, 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00147386, 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033073+0000", 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9999999999999998e-07 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033078+0000", 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5430000000000001e-06 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033087+0000", 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5420000000000005e-06 2024-09-14T22:53:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033093+0000", 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9479999999999996e-06 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033185+0000", 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2298000000000003e-05 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033218+0000", 2024-09-14T22:53:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3389999999999997e-05 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034501+0000", 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012830160000000001 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034546+0000", 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4623000000000003e-05 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77389 3.9 e40/24 3:95bf6a49:::rbd_data.113564f98e71.0000000000000053:head v 40'64, mlcod=40'60)", 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.040338+0000", 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 17.652317358000001, 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0009955369999999999, 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040338+0000", 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040338+0000", 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040339+0000", 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.403e-06 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040348+0000", 2024-09-14T22:53:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2329999999999995e-06 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040364+0000", 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6137000000000002e-05 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040372+0000", 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1890000000000007e-06 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040435+0000", 2024-09-14T22:53:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2416999999999995e-05 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.040456+0000", 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1058000000000001e-05 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.041290+0000", 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00083392600000000002 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.041333+0000", 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3173999999999997e-05 2024-09-14T22:53:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 17.484133427, 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:53:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:78175 3.4 e41/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 41'105, mlcod=40'102)", 2024-09-14T22:53:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.726854473, 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0049595489999999997, 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965801+0000", 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.35e-07 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965834+0000", 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3683999999999998e-05 2024-09-14T22:53:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965846+0000", 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1620999999999999e-05 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965855+0000", 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1130000000000007e-06 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965939+0000", 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4029000000000005e-05 2024-09-14T22:53:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965963+0000", 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3529000000000001e-05 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970705+0000", 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.004742853 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970760+0000", 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4484999999999998e-05 2024-09-14T22:53:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPush(3.1e 45/44 [PushOp(3:7c4423e3:::rbd_data.113564f98e71.0000000000000010:b, version: 40'64, data_included: [0~12288,14336~22528,38912~22528,63488~198656], data_size: 256000, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:7c4423e3:::rbd_data.113564f98e71.0000000000000010:b@40'64, size: 262144, copy_subset: [(0, 262144)], clone_subset: {}, snapset: b=[]:{b: [b]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262144, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:32.810280+0000", 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 11.882374442, 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010089669999999999, 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810280+0000", 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810280+0000", 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810281+0000", 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1099999999999995e-07 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810505+0000", 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022378099999999999 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810522+0000", 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6693000000000001e-05 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810539+0000", 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7309e-05 2024-09-14T22:53:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810629+0000", 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9928999999999999e-05 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.810643+0000", 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4277000000000001e-05 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:32.811289+0000", 2024-09-14T22:53:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00064626699999999996 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:116229 3.1e e48/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 48'99, mlcod=48'97)", 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.3885625719999997, 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034125309999999999, 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304095+0000", 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3029999999999998e-06 2024-09-14T22:53:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304126+0000", 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1374999999999997e-05 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304140+0000", 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4518000000000001e-05 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304150+0000", 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304213+0000", 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3366000000000003e-05 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304237+0000", 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4379000000000002e-05 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307471+0000", 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032338829999999999 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307505+0000", 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3552e-05 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.3604716530000003, 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:53:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:131041 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'121, mlcod=52'119)", 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.469601+0000", 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.2230541959999996, 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012083899999999999, 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469601+0000", 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469601+0000", 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469602+0000", 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2309999999999999e-06 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469641+0000", 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8751000000000002e-05 2024-09-14T22:53:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469655+0000", 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4299e-05 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469661+0000", 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5470000000000002e-06 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469706+0000", 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4747999999999999e-05 2024-09-14T22:53:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.469722+0000", 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.5287000000000001e-05 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.470782+0000", 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001060273 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.470809+0000", 2024-09-14T22:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7254e-05 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:131066 3.1e e52/44 3:7b98fc53:::rbd_data.1135a9fbf9fc.000000000000000f:head v 52'143, mlcod=52'141)", 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.503635+0000", 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.1890199670000001, 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010888740000000001, 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503635+0000", 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503635+0000", 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503637+0000", 2024-09-14T22:53:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.84e-06 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503700+0000", 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3157000000000005e-05 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503718+0000", 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7994000000000001e-05 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503743+0000", 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5208999999999999e-05 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503822+0000", 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9328999999999999e-05 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.503847+0000", 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4624e-05 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.504687+0000", 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000840032 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.504724+0000", 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6689000000000001e-05 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:131796 3.1e e53/44 3:79e10459:::rbd_data.1135a5fb387.0000000000000058:head v 53'145, mlcod=52'143)", 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.1312079769999999, 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012088559999999999, 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8e-07 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561522+0000", 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4567999999999995e-05 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561534+0000", 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2517999999999999e-05 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561546+0000", 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1440000000000001e-05 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561630+0000", 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4615000000000002e-05 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561659+0000", 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8969e-05 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.562606+0000", 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00094671999999999996 2024-09-14T22:53:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.562656+0000", 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9845999999999997e-05 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:132545 3.4 e53/24 3:21a8123c:::rbd_data.1135a5fb387.000000000000004a:head v 53'133, mlcod=43'131)", 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:41.642699+0000", 2024-09-14T22:53:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.0499558609999999, 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010215070000000001, 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.642699+0000", 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.642699+0000", 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.642701+0000", 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6080000000000001e-06 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.642714+0000", 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3134e-05 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.642729+0000", 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4827000000000001e-05 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.642735+0000", 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6819999999999997e-06 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.642804+0000", 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8524999999999999e-05 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.642824+0000", 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0588999999999999e-05 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.643666+0000", 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00084135299999999998 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.643721+0000", 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4789e-05 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:133036 3.4 e54/24 3:22b239af:::rbd_data.1135a4365f9c.0000000000000001:head v 54'137, mlcod=53'135)", 2024-09-14T22:53:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 2.4227717040000001, 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002821278, 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7100000000000001e-07 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270380+0000", 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00049641500000000005 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270393+0000", 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3088000000000001e-05 2024-09-14T22:53:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270402+0000", 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6199999999999994e-06 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270470+0000", 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7459999999999994e-05 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270498+0000", 2024-09-14T22:53:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8214000000000001e-05 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272657+0000", 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021591499999999999 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272704+0000", 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7160000000000002e-05 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:44.783 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:44.784 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:44.784 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:44.782+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:44.884 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:44.883+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:44.904 INFO:teuthology.orchestra.run.smithi076.stderr:set 3.8 pg_upmap mapping to [3,6] 2024-09-14T22:53:44.913 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:53:44.936 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:44.936 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:44.936 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:44.936 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:44.945 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:44.971 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone15 from 0xcb527d4 (overlap 0x30c3d2) to 0x9c0c3 2024-09-14T22:53:44.985 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:44.984+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:45.027 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T22:53:45.028 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:45.028 INFO:teuthology.orchestra.run.smithi076.stdout:1465 read 0xd099b0 thru 0xd0a9c9 (0x101a bytes) 2024-09-14T22:53:45.029 INFO:teuthology.orchestra.run.smithi076.stdout:1467 punch from 0x57da2fb to 0x57dabea, (0x8ef bytes) 2024-09-14T22:53:45.032 INFO:teuthology.orchestra.run.smithi076.stdout:1469 writesame 0xac4fdcb thru 0xac523ab data_size 0x1(0x25e1 bytes) 2024-09-14T22:53:45.081 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:45.082 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:45.082 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:45.082 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:45.082 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:45.086 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:45.085+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:45.091 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:45.187 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:45.185+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:45.197 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:45.197 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:45.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30305 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1349632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.291044847999999, 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051104002000000003, 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.198 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30305 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904380+0000", 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8799999999999998e-07 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904388+0000", 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7000000000000008e-06 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904395+0000", 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904401+0000", 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.147e-06 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955309+0000", 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.050908627999999997 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955388+0000", 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8615999999999994e-05 2024-09-14T22:53:45.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955484+0000", 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6323000000000006e-05 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.291017338, 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.202 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:45.206 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290996801999999, 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:45.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:53:45.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:53:45.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290966803, 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.211 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:53:45.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:53:45.215 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290935650000002, 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:45.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290912993999999, 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:53:45.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290892493000001, 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.225 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:53:45.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:45.229 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290872732, 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:45.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:53:45.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290851947, 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.234 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.238 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290819535000001, 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:53:45.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:53:45.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290798135999999, 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:45.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:53:45.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:53:45.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290736737, 2024-09-14T22:53:45.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:53:45.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.252 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290696824000001, 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.253 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:53:45.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290667068000001, 2024-09-14T22:53:45.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:53:45.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:53:45.261 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290636833000001, 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.262 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:53:45.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290612094, 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:53:45.266 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:53:45.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290588664000001, 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:53:45.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290492927999999, 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:53:45.275 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.279 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290464006000001, 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:45.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 23.290440681, 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:45.284 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:53:45.288 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.289 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:45.289 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:45.289 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:45.291 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:45.291 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:45.289+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:45.391 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:45.390+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:45.393 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:45.393 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:45.393 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:45.394 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:45.402 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:45.493 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:45.491+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:45.545 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:45.546 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:45.546 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:45.546 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:45.546 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:45.554 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:45.597 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:45.596+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:45.657 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116113 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5611520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6014407360000007, 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053031104000000003, 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116113 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:53:45.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053986+0000", 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010658 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053989+0000", 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3579999999999999e-06 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053998+0000", 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8319999999999995e-06 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054004+0000", 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3579999999999999e-06 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106732+0000", 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052727505000000001 2024-09-14T22:53:45.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106816+0000", 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4482999999999997e-05 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106910+0000", 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3987999999999995e-05 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6013050379999996, 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:53:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6011099459999993, 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:53:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6009571559999998, 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:53:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:53:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:53:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:53:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:53:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:53:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:53:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6007521340000004, 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:53:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:53:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6004814679999999, 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:53:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:53:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6002193079999998, 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:53:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:53:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6000604559999996, 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:53:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:53:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5998736600000001, 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:53:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:53:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5996714149999995, 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:53:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5994630569999995, 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:53:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5991493400000003, 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:53:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5989435780000001, 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:53:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.598774702, 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:53:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5985040080000008, 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:53:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:53:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5983547330000007, 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5981134909999994, 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:53:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116130 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5566464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5978924570000004, 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053025477000000001, 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116130 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057631+0000", 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020390499999999999 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057636+0000", 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4299999999999999e-06 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057647+0000", 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1516e-05 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057653+0000", 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3620000000000002e-06 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110336+0000", 2024-09-14T22:53:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052682584999999997 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110379+0000", 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2901000000000001e-05 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110453+0000", 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3777999999999997e-05 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116131 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5685248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5976549440000003, 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053011547999999999, 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116131 2024-09-14T22:53:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057795+0000", 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013012700000000001 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057805+0000", 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0482999999999999e-05 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057817+0000", 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1499e-05 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057827+0000", 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0091999999999999e-05 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110493+0000", 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052666339999999999 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110578+0000", 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4876000000000005e-05 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110676+0000", 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8130999999999999e-05 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.289423459, 2024-09-14T22:53:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:53:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:53:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:53:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:53:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:45.750 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:45.751 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:45.752 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:45.750+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:45.852 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:45.851+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:45.894 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:45.894 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:45.894 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:45.894 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:45.902 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:45.907 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:53:45.953 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:45.952+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:46.005 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:46.006 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:46.006 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:46.006 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:46.006 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:46.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:46.054 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:46.053+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:46.155 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:46.154+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.259929282, 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:53:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127349 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4120576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6342397260000006, 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053205084, 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127349 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7000000000000005e-07 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521644+0000", 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521715+0000", 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0736999999999997e-05 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521725+0000", 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0237999999999999e-05 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574705+0000", 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052979928000000003 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574768+0000", 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2618000000000003e-05 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574846+0000", 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8143e-05 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6341351629999998, 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:53:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6340471710000006, 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:53:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6338132539999997, 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:53:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6336323589999999, 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:53:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.187 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6333328559999991, 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.188 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:53:46.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:53:46.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127361 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6330127099999991, 2024-09-14T22:53:46.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053260788000000003, 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127361 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:53:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522869+0000", 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522872+0000", 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.066e-06 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522879+0000", 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8110000000000003e-06 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522885+0000", 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.003e-06 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575959+0000", 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073515000000002 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576037+0000", 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7991999999999995e-05 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576129+0000", 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1772000000000002e-05 2024-09-14T22:53:46.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6329866459999991, 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:53:46.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.201 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6326229179999991, 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:53:46.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:53:46.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6320333009999999, 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:53:46.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.210 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.631831064, 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:53:46.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:53:46.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6316412529999997, 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.219 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6313292500000003, 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:53:46.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:53:46.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:53:46.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6310867630000008, 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.228 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6309278010000003, 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:53:46.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:53:46.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6302935309999995, 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:53:46.233 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:53:46.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:53:46.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.237 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6300794209999996, 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.238 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:53:46.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:53:46.242 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.6299429510000003, 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:53:46.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:53:46.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.629690471, 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.247 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:53:46.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:53:46.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.251 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:46.252 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:46.293 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:46.292+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:46.357 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:46.356+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:46.395 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:46.395 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:46.403 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:46.458 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:46.457+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:46.505 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:46.505 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:46.505 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:46.505 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:46.505 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:46.513 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:46.558 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:46.557+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.534190840000001, 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:53:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:53:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:53:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:53:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.533972943999999, 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.531289298000001, 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.531130882999999, 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:53:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:53:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.531103616999999, 2024-09-14T22:53:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:53:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:53:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.530896475999999, 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:53:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:53:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.530862211999999, 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:53:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:53:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.529230166000001, 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:53:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.529205554000001, 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:53:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:53:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.529071267999999, 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:53:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:53:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.528021559999999, 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:53:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.527919375, 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:53:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.527895471000001, 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.527861950999998, 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:53:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:53:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.527707305, 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:53:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.527666131, 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:53:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.527637971000001, 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:53:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.527448580000001, 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:53:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:53:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.611358753000001, 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:53:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.611338922000002, 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:46.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:46.708 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:46.709 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:46.710 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:46.708+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:46.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:46.810 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:46.809+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:46.844 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:46.844 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:46.854 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:46.855 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:46.861 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:46.862 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:46.863 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:46.878 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:46.878 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:46.893 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:46.911 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:46.910+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:46.912 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:46.912 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:46.929 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:46.929 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:46.946 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:46.946 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:46.964 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:46.964 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:46.970 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:46.970 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:46.979 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:47.022 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:47.021+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:47.081 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:47.081 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:47.089 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:47.130 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:47.129+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:47.194 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:47.194 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:47.202 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:47.214 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:47.213+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:47.304 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:47.304 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:47.313 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:47.314 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:47.314+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:47.415 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:47.414+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:47.416 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:47.416 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:47.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:47.516 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:47.514+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:47.568 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:47.568 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:47.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:47.617 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:47.616+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:47.718 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:47.717+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:47.718 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:47.719 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:47.727 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:47.758 INFO:teuthology.orchestra.run.smithi076.stdout:1470 write 0x3c13eac thru 0x3c1b04a (0x719f bytes) 2024-09-14T22:53:47.765 INFO:teuthology.orchestra.run.smithi076.stdout:1471 punch from 0x234a547 to 0x23518d4, (0x738d bytes) 2024-09-14T22:53:47.765 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:47.766 INFO:teuthology.orchestra.run.smithi076.stdout:1476 read 0xa7c331e thru 0xa7ccbba (0x989d bytes) 2024-09-14T22:53:47.766 INFO:teuthology.orchestra.run.smithi076.stdout:1477 read 0x645a291 thru 0x645a41f (0x18f bytes) 2024-09-14T22:53:47.766 INFO:teuthology.orchestra.run.smithi076.stdout:1478 read 0x281f0c4 thru 0x28239db (0x4918 bytes) 2024-09-14T22:53:47.772 INFO:teuthology.orchestra.run.smithi076.stdout:1479 write 0xcff4f03 thru 0xd001a0f (0xcb0d bytes) 2024-09-14T22:53:47.778 INFO:teuthology.orchestra.run.smithi076.stdout:1481 write 0xa3ae721 thru 0xa3b7d85 (0x9665 bytes) 2024-09-14T22:53:47.785 INFO:teuthology.orchestra.run.smithi076.stdout:1482 write 0xbd07291 thru 0xbd14d1d (0xda8d bytes) 2024-09-14T22:53:47.796 INFO:teuthology.orchestra.run.smithi076.stdout:1483 write 0x2f27ded thru 0x2f314c4 (0x96d8 bytes) 2024-09-14T22:53:47.812 INFO:teuthology.orchestra.run.smithi076.stdout:1485 writesame 0xd69cb97 thru 0xd6a6248 data_size 0x1(0x96b2 bytes) 2024-09-14T22:53:47.812 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:47.812 INFO:teuthology.orchestra.run.smithi076.stdout:1487 write 0x96551be thru 0x966121c (0xc05f bytes) 2024-09-14T22:53:47.818 INFO:teuthology.orchestra.run.smithi076.stdout:1488 writesame 0x88586a2 thru 0x8862eb9 data_size 0x21(0xa818 bytes) 2024-09-14T22:53:47.818 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:47.817+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:47.823 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:47.824 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:47.831 INFO:teuthology.orchestra.run.smithi076.stdout:1491 trunc from 0xd6a6249 to 0x6bb08ca 2024-09-14T22:53:47.857 INFO:teuthology.orchestra.run.smithi076.stdout:1493 write 0x282c8fc thru 0x283a004 (0xd709 bytes) 2024-09-14T22:53:47.861 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:47.861 INFO:teuthology.orchestra.run.smithi076.stdout:1495 write 0x55f8438 thru 0x55f8f6e (0xb37 bytes) 2024-09-14T22:53:47.864 INFO:teuthology.orchestra.run.smithi076.stdout:1497 read 0x2073293 thru 0x207da9a (0xa808 bytes) 2024-09-14T22:53:47.864 INFO:teuthology.orchestra.run.smithi076.stdout:1498 read 0xde0bcb thru 0xdea3e8 (0x981e bytes) 2024-09-14T22:53:47.870 INFO:teuthology.orchestra.run.smithi076.stdout:1500 write 0x6f2de50 thru 0x6f346e0 (0x6891 bytes) 2024-09-14T22:53:47.876 INFO:teuthology.orchestra.run.smithi076.stdout:1501 punch from 0x48fdf3c to 0x4900e0d, (0x2ed1 bytes) 2024-09-14T22:53:47.876 INFO:teuthology.orchestra.run.smithi076.stdout:1503 punch from 0x6466cac to 0x646bcd5, (0x5029 bytes) 2024-09-14T22:53:47.876 INFO:teuthology.orchestra.run.smithi076.stdout:1504 write 0x3d6fc59 thru 0x3d751d1 (0x5579 bytes) 2024-09-14T22:53:47.879 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:47.880 INFO:teuthology.orchestra.run.smithi076.stdout:1506 read 0x670bf77 thru 0x67122a7 (0x6331 bytes) 2024-09-14T22:53:47.880 INFO:teuthology.orchestra.run.smithi076.stdout:truncating to largest ever: 0xed20ab6 2024-09-14T22:53:47.889 INFO:teuthology.orchestra.run.smithi076.stdout:1507 trunc from 0x6f346e1 to 0xed20ab6 2024-09-14T22:53:47.895 INFO:teuthology.orchestra.run.smithi076.stdout:1508 read 0x532b75f thru 0x533a505 (0xeda7 bytes) 2024-09-14T22:53:47.895 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:47.895 INFO:teuthology.orchestra.run.smithi076.stdout:1511 writesame 0xaefbbe7 thru 0xaefc1b6 data_size 0x4(0x5d0 bytes) 2024-09-14T22:53:47.898 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:47.899 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:47.899 INFO:teuthology.orchestra.run.smithi076.stdout:1512 write 0x13ce81c thru 0x13cfe77 (0x165c bytes) 2024-09-14T22:53:47.905 INFO:teuthology.orchestra.run.smithi076.stdout:1513 read 0x2d52a8b thru 0x2d54ebc (0x2432 bytes) 2024-09-14T22:53:47.906 INFO:teuthology.orchestra.run.smithi076.stdout:1514 read 0x6599961 thru 0x659d4cd (0x3b6d bytes) 2024-09-14T22:53:47.906 INFO:teuthology.orchestra.run.smithi076.stdout:1515 read 0xaa8fd9e thru 0xaa967c8 (0x6a2b bytes) 2024-09-14T22:53:47.906 INFO:teuthology.orchestra.run.smithi076.stdout:1516 read 0x2960f86 thru 0x2965583 (0x45fe bytes) 2024-09-14T22:53:47.907 INFO:teuthology.orchestra.run.smithi076.stdout:1517 read 0x456986f thru 0x4570e43 (0x75d5 bytes) 2024-09-14T22:53:47.910 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:47.915 INFO:teuthology.orchestra.run.smithi076.stdout:1518 trunc from 0xed20ab6 to 0x86e9e41 2024-09-14T22:53:47.919 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:47.918+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:47.922 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:53:47.928 INFO:teuthology.orchestra.run.smithi076.stdout:1520 write 0xb47ed6 thru 0xb5350a (0xb635 bytes) 2024-09-14T22:53:47.985 INFO:teuthology.orchestra.run.smithi076.stdout:1521 write 0xc3ff3c1 thru 0xc405cc5 (0x6905 bytes) 2024-09-14T22:53:47.992 INFO:teuthology.orchestra.run.smithi076.stdout:1522 read 0x87c32ad thru 0x87ccb9d (0x98f1 bytes) 2024-09-14T22:53:47.992 INFO:teuthology.orchestra.run.smithi076.stdout:1524 read 0x931b2ad thru 0x931ee0f (0x3b63 bytes) 2024-09-14T22:53:47.999 INFO:teuthology.orchestra.run.smithi076.stdout:1525 writesame 0xd3a0156 thru 0xd3a766b data_size 0x7(0x7516 bytes) 2024-09-14T22:53:47.999 INFO:teuthology.orchestra.run.smithi076.stdout:1526 read 0xc04bbc3 thru 0xc04fcf3 (0x4131 bytes) 2024-09-14T22:53:47.999 INFO:teuthology.orchestra.run.smithi076.stdout:1527 read 0x1c2a330 thru 0x1c347fb (0xa4cc bytes) 2024-09-14T22:53:47.999 INFO:teuthology.orchestra.run.smithi076.stdout:1529 write 0x7273826 thru 0x727b58b (0x7d66 bytes) 2024-09-14T22:53:48.011 INFO:teuthology.orchestra.run.smithi076.stdout:1534 write 0xea7ce2e thru 0xea86655 (0x9828 bytes) 2024-09-14T22:53:48.020 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:48.019+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:48.021 INFO:teuthology.orchestra.run.smithi076.stdout:1535 write 0xebd4f84 thru 0xebdb6f7 (0x6774 bytes) 2024-09-14T22:53:48.025 INFO:teuthology.orchestra.run.smithi076.stdout:1536 write 0x6194f06 thru 0x61a3b68 (0xec63 bytes) 2024-09-14T22:53:48.027 INFO:teuthology.orchestra.run.smithi076.stdout:1538 read 0x23f2ec9 thru 0x2401091 (0xe1c9 bytes) 2024-09-14T22:53:48.027 INFO:teuthology.orchestra.run.smithi076.stdout:1539 punch from 0xad7de66 to 0xad80fc1, (0x315b bytes) 2024-09-14T22:53:48.027 INFO:teuthology.orchestra.run.smithi076.stdout:1540 write 0x9a6f1b4 thru 0x9a7db2e (0xe97b bytes) 2024-09-14T22:53:48.034 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:48.034 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:48.035 INFO:teuthology.orchestra.run.smithi076.stdout:1541 write 0x9d849bf thru 0x9d8e9d3 (0xa015 bytes) 2024-09-14T22:53:48.039 INFO:teuthology.orchestra.run.smithi076.stdout:1544 write 0x6161021 thru 0x616d25b (0xc23b bytes) 2024-09-14T22:53:48.040 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:48.040 INFO:teuthology.orchestra.run.smithi076.stdout:1546 read 0x30a587b thru 0x30a7cac (0x2432 bytes) 2024-09-14T22:53:48.041 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:48.041 INFO:teuthology.orchestra.run.smithi076.stdout:1549 write 0xe543b6a thru 0xe546fde (0x3475 bytes) 2024-09-14T22:53:48.041 INFO:teuthology.orchestra.run.smithi076.stdout:1550 read 0x22d6c4 thru 0x238b5b (0xb498 bytes) 2024-09-14T22:53:48.042 INFO:teuthology.orchestra.run.smithi076.stdout:1551 punch from 0x6a0d9cf to 0x6a1516c, (0x779d bytes) 2024-09-14T22:53:48.043 INFO:teuthology.orchestra.run.smithi076.stdout:1552 read 0x8d8dcaa thru 0x8d9c52d (0xe884 bytes) 2024-09-14T22:53:48.043 INFO:teuthology.orchestra.run.smithi076.stdout:1554 read 0x266e8d0 thru 0x267d9e8 (0xf119 bytes) 2024-09-14T22:53:48.046 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:48.049 INFO:teuthology.orchestra.run.smithi076.stdout:1555 trunc from 0xebdb6f8 to 0x9febc1f 2024-09-14T22:53:48.061 INFO:teuthology.orchestra.run.smithi076.stdout:1556 write 0x6b2a5f6 thru 0x6b386d5 (0xe0e0 bytes) 2024-09-14T22:53:48.062 INFO:teuthology.orchestra.run.smithi076.stdout:1559 write 0x9a75ef3 thru 0x9a7d0e1 (0x71ef bytes) 2024-09-14T22:53:48.065 INFO:teuthology.orchestra.run.smithi076.stdout:1561 read 0x841f1f3 thru 0x84219d4 (0x27e2 bytes) 2024-09-14T22:53:48.065 INFO:teuthology.orchestra.run.smithi076.stdout:1562 read 0x3eaaaf8 thru 0x3eaade9 (0x2f2 bytes) 2024-09-14T22:53:48.065 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:48.065 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:48.065 INFO:teuthology.orchestra.run.smithi076.stdout:1565 writesame 0x7b4a8aa thru 0x7b54969 data_size 0x20(0xa0c0 bytes) 2024-09-14T22:53:48.083 INFO:teuthology.orchestra.run.smithi076.stdout:1566 trunc from 0x9febc1f to 0x19cca89 2024-09-14T22:53:48.109 INFO:teuthology.orchestra.run.smithi076.stdout:1567 read 0x118cc36 thru 0x1197c1e (0xafe9 bytes) 2024-09-14T22:53:48.117 INFO:teuthology.orchestra.run.smithi076.stdout:1569 writesame 0x2e42427 thru 0x2e44ed3 data_size 0x19(0x2aad bytes) 2024-09-14T22:53:48.121 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:48.120+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:48.135 INFO:teuthology.orchestra.run.smithi076.stdout:1570 writesame 0xdb109a6 thru 0xdb1753c data_size 0x1(0x6b97 bytes) 2024-09-14T22:53:48.135 INFO:teuthology.orchestra.run.smithi076.stdout:1572 read 0x3db4ffc thru 0x3db9c90 (0x4c95 bytes) 2024-09-14T22:53:48.135 INFO:teuthology.orchestra.run.smithi076.stdout:1573 writesame 0x99bef17 thru 0x99ca325 data_size 0x23(0xb40f bytes) 2024-09-14T22:53:48.148 INFO:teuthology.orchestra.run.smithi076.stdout:1575 writesame 0xe56a9fa thru 0xe571379 data_size 0x40(0x6980 bytes) 2024-09-14T22:53:48.148 INFO:teuthology.orchestra.run.smithi076.stdout:1576 read 0x6ba3bc thru 0x6bba7c (0x16c1 bytes) 2024-09-14T22:53:48.148 INFO:teuthology.orchestra.run.smithi076.stdout:1577 punch from 0xcd71cba to 0xcd763a2, (0x46e8 bytes) 2024-09-14T22:53:48.149 INFO:teuthology.orchestra.run.smithi076.stdout:1578 read 0x7242ae7 thru 0x7243547 (0xa61 bytes) 2024-09-14T22:53:48.149 INFO:teuthology.orchestra.run.smithi076.stdout:1579 write 0x5fb7e0b thru 0x5fbe4d6 (0x66cc bytes) 2024-09-14T22:53:48.152 INFO:teuthology.orchestra.run.smithi076.stdout:1580 clone 17 order 24 su 524288 sc 7 2024-09-14T22:53:48.166 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:48.166 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:48.175 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:48.259 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:48.258+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:48.319 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:48.319 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:48.322 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:48.321+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:48.328 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:48.423 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:48.422+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:48.472 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:48.472 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:48.481 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:48.526 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:48.525+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:48.584 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:48.584 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:48.594 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:48.635 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:48.634+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:48.726 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:48.725+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:48.739 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:48.739 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:48.748 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:48.826 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:48.825+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:48.858 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:48.859 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:48.867 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:48.919 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:53:48.927 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:48.926+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:48.976 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:48.999 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone16 from 0xe57137a (overlap 0x19cca89) to 0x0 2024-09-14T22:53:49.028 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:49.026+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:49.029 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:49.029 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:49.039 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:49.111 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:49.111 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:49.129 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:49.128+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:49.130 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:49.130 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:49.147 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:49.147 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:49.148 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T22:53:49.152 INFO:teuthology.orchestra.run.smithi076.stdout:1582 write 0x329e6a7 thru 0x32ac972 (0xe2cc bytes) 2024-09-14T22:53:49.153 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:49.153 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:49.164 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:49.165 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:49.165 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:49.179 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.179 INFO:teuthology.orchestra.run.smithi076.stdout:1584 write 0x51f4f1f thru 0x51fdd3e (0x8e20 bytes) 2024-09-14T22:53:49.184 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:49.184 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:49.200 INFO:teuthology.orchestra.run.smithi076.stdout:1585 read 0x3a80a0b thru 0x3a8acf7 (0xa2ed bytes) 2024-09-14T22:53:49.200 INFO:teuthology.orchestra.run.smithi076.stdout:1586 read 0xc6b63d6 thru 0xc6c54ed (0xf118 bytes) 2024-09-14T22:53:49.200 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:49.201 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:49.208 INFO:teuthology.orchestra.run.smithi076.stdout:1587 trunc from 0xe57137a to 0x78b6025 2024-09-14T22:53:49.220 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:49.228 INFO:teuthology.orchestra.run.smithi076.stdout:1588 trunc from 0x78b6025 to 0xa225eec 2024-09-14T22:53:49.230 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:49.229+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:49.237 INFO:teuthology.orchestra.run.smithi076.stdout:1590 trunc from 0xa225eec to 0x68ba8a6 2024-09-14T22:53:49.238 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:49.238 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:49.280 INFO:teuthology.orchestra.run.smithi076.stdout:1591 trunc from 0x68ba8a6 to 0x4b997a2 2024-09-14T22:53:49.285 INFO:teuthology.orchestra.run.smithi076.stdout:1593 read 0x20c57c3 thru 0x20cb795 (0x5fd3 bytes) 2024-09-14T22:53:49.285 INFO:teuthology.orchestra.run.smithi076.stdout:1594 write 0x4a4e527 thru 0x4a5355a (0x5034 bytes) 2024-09-14T22:53:49.290 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:49.290 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:49.299 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:49.300 INFO:teuthology.orchestra.run.smithi076.stdout:1596 writesame 0xabcbc6e thru 0xabcbf1f data_size 0x2(0x2b2 bytes) 2024-09-14T22:53:49.305 INFO:teuthology.orchestra.run.smithi076.stdout:1599 write 0xb67048b thru 0xb67bc22 (0xb798 bytes) 2024-09-14T22:53:49.308 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.308 INFO:teuthology.orchestra.run.smithi076.stdout:1601 read 0x63a98c3 thru 0x63af0aa (0x57e8 bytes) 2024-09-14T22:53:49.308 INFO:teuthology.orchestra.run.smithi076.stdout:1602 read 0xc2c159 thru 0xc2f78f (0x3637 bytes) 2024-09-14T22:53:49.310 INFO:teuthology.orchestra.run.smithi076.stdout:1603 trunc from 0xb67bc23 to 0xcbfbe89 2024-09-14T22:53:49.316 INFO:teuthology.orchestra.run.smithi076.stdout:1606 writesame 0xc23cad thru 0xc28414 data_size 0x14(0x4768 bytes) 2024-09-14T22:53:49.326 INFO:teuthology.orchestra.run.smithi076.stdout:1607 write 0xebfa9ca thru 0xec038b5 (0x8eec bytes) 2024-09-14T22:53:49.330 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.330 INFO:teuthology.orchestra.run.smithi076.stdout:1609 write 0xae3f49d thru 0xae41fb6 (0x2b1a bytes) 2024-09-14T22:53:49.333 INFO:teuthology.orchestra.run.smithi076.stdout:1610 write 0x51d0ba thru 0x52c6a4 (0xf5eb bytes) 2024-09-14T22:53:49.334 INFO:teuthology.orchestra.run.smithi076.stdout:1612 write 0xf55839 thru 0xf5cbdb (0x73a3 bytes) 2024-09-14T22:53:49.336 INFO:teuthology.orchestra.run.smithi076.stdout:1613 writesame 0xd70de35 thru 0xd718d43 data_size 0x1(0xaf0f bytes) 2024-09-14T22:53:49.343 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:49.342+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:49.361 INFO:teuthology.orchestra.run.smithi076.stdout:1614 punch from 0x33f8351 to 0x3400a3b, (0x86ea bytes) 2024-09-14T22:53:49.361 INFO:teuthology.orchestra.run.smithi076.stdout:1615 writesame 0xe92325b thru 0xe924421 data_size 0x1(0x11c7 bytes) 2024-09-14T22:53:49.367 INFO:teuthology.orchestra.run.smithi076.stdout:1616 punch from 0x9df3f95 to 0x9e03754, (0xf7bf bytes) 2024-09-14T22:53:49.368 INFO:teuthology.orchestra.run.smithi076.stdout:1617 read 0x2472ac1 thru 0x2474dce (0x230e bytes) 2024-09-14T22:53:49.368 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.368 INFO:teuthology.orchestra.run.smithi076.stdout:1620 read 0x547be1a thru 0x548523e (0x9425 bytes) 2024-09-14T22:53:49.368 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.369 INFO:teuthology.orchestra.run.smithi076.stdout:1624 read 0x87a7320 thru 0x87b3485 (0xc166 bytes) 2024-09-14T22:53:49.369 INFO:teuthology.orchestra.run.smithi076.stdout:1625 read 0x6befd3 thru 0x6c17e2 (0x2810 bytes) 2024-09-14T22:53:49.369 INFO:teuthology.orchestra.run.smithi076.stdout:1626 write 0xb3b7436 thru 0xb3bbe6c (0x4a37 bytes) 2024-09-14T22:53:49.372 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.375 INFO:teuthology.orchestra.run.smithi076.stdout:1628 trunc from 0xec038b6 to 0xc2b7027 2024-09-14T22:53:49.388 INFO:teuthology.orchestra.run.smithi076.stdout:1629 read 0x7af3910 thru 0x7b01ea3 (0xe594 bytes) 2024-09-14T22:53:49.388 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.389 INFO:teuthology.orchestra.run.smithi076.stdout:1632 read 0x6e45142 thru 0x6e52cba (0xdb79 bytes) 2024-09-14T22:53:49.389 INFO:teuthology.orchestra.run.smithi076.stdout:1633 write 0x1d116ee thru 0x1d192c4 (0x7bd7 bytes) 2024-09-14T22:53:49.390 INFO:teuthology.orchestra.run.smithi076.stdout:1635 writesame 0x88b587d thru 0x88bd1ca data_size 0x1(0x794e bytes) 2024-09-14T22:53:49.410 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:49.410 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:49.410 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:49.410 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:49.410 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:49.410 INFO:teuthology.orchestra.run.smithi076.stdout:1637 punch from 0x344fb73 to 0x3454480, (0x490d bytes) 2024-09-14T22:53:49.412 INFO:teuthology.orchestra.run.smithi076.stdout:1638 trunc from 0xc2b7027 to 0xa993996 2024-09-14T22:53:49.419 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:49.420 INFO:teuthology.orchestra.run.smithi076.stdout:1640 read 0x19656e7 thru 0x1966a98 (0x13b2 bytes) 2024-09-14T22:53:49.420 INFO:teuthology.orchestra.run.smithi076.stdout:1642 read 0xef0227 thru 0xefebd7 (0xe9b1 bytes) 2024-09-14T22:53:49.421 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.421 INFO:teuthology.orchestra.run.smithi076.stdout:1644 read 0x3d1e427 thru 0x3d1e8da (0x4b4 bytes) 2024-09-14T22:53:49.421 INFO:teuthology.orchestra.run.smithi076.stdout:1645 punch from 0x3d701e6 to 0x3d7f17d, (0xef97 bytes) 2024-09-14T22:53:49.429 INFO:teuthology.orchestra.run.smithi076.stdout:1646 writesame 0xca86cb6 thru 0xca93be3 data_size 0x17(0xcf2e bytes) 2024-09-14T22:53:49.429 INFO:teuthology.orchestra.run.smithi076.stdout:1647 write 0x17e02d1 thru 0x17e739e (0x70ce bytes) 2024-09-14T22:53:49.431 INFO:teuthology.orchestra.run.smithi076.stdout:1649 write 0x93b7105 thru 0x93c1e3a (0xad36 bytes) 2024-09-14T22:53:49.431 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:49.430+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:49.433 INFO:teuthology.orchestra.run.smithi076.stdout:1650 read 0x67c0f51 thru 0x67cccb4 (0xbd64 bytes) 2024-09-14T22:53:49.434 INFO:teuthology.orchestra.run.smithi076.stdout:1651 writesame 0x9e276c0 thru 0x9e340c2 data_size 0x1(0xca03 bytes) 2024-09-14T22:53:49.468 INFO:teuthology.orchestra.run.smithi076.stdout:1652 write 0xe8312bd thru 0xe835966 (0x46aa bytes) 2024-09-14T22:53:49.472 INFO:teuthology.orchestra.run.smithi076.stdout:1654 punch from 0xc350f65 to 0xc351a86, (0xb21 bytes) 2024-09-14T22:53:49.485 INFO:teuthology.orchestra.run.smithi076.stdout:1655 trunc from 0xe835967 to 0x3b125ba 2024-09-14T22:53:49.498 INFO:teuthology.orchestra.run.smithi076.stdout:1656 read 0x2ff722c thru 0x2ffbd86 (0x4b5b bytes) 2024-09-14T22:53:49.498 INFO:teuthology.orchestra.run.smithi076.stdout:1657 punch from 0x3844020 to 0x384b44a, (0x742a bytes) 2024-09-14T22:53:49.504 INFO:teuthology.orchestra.run.smithi076.stdout:1658 trunc from 0x3b125ba to 0x82b3554 2024-09-14T22:53:49.515 INFO:teuthology.orchestra.run.smithi076.stdout:1659 trunc from 0x82b3554 to 0x34659bf 2024-09-14T22:53:49.524 INFO:teuthology.orchestra.run.smithi076.stdout:1660 read 0x26a4160 thru 0x26abb1e (0x79bf bytes) 2024-09-14T22:53:49.532 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:49.531+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:49.538 INFO:teuthology.orchestra.run.smithi076.stdout:1661 write 0x94515fb thru 0x945d92b (0xc331 bytes) 2024-09-14T22:53:49.546 INFO:teuthology.orchestra.run.smithi076.stdout:1662 trunc from 0x945d92c to 0x497b41c 2024-09-14T22:53:49.559 INFO:teuthology.orchestra.run.smithi076.stdout:1663 read 0x1b7bcd4 thru 0x1b81a83 (0x5db0 bytes) 2024-09-14T22:53:49.571 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:53:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:53:49.571 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.672779617, 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:53:49.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:53:49.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:53:49.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127349 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4120576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.047090061, 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053205084, 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127349 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7000000000000005e-07 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521644+0000", 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521715+0000", 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0736999999999997e-05 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521725+0000", 2024-09-14T22:53:49.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0237999999999999e-05 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574705+0000", 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052979928000000003 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574768+0000", 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2618000000000003e-05 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574846+0000", 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8143e-05 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.046985498, 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.046897506000001, 2024-09-14T22:53:49.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:53:49.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.591 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.046663589, 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:53:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:53:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.046482694, 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.046183191000001, 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127361 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.045863045000001, 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053260788000000003, 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127361 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522869+0000", 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522872+0000", 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.066e-06 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522879+0000", 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8110000000000003e-06 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522885+0000", 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.003e-06 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575959+0000", 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073515000000002 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576037+0000", 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7991999999999995e-05 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576129+0000", 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1772000000000002e-05 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.045836981000001, 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:53:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.045473253000001, 2024-09-14T22:53:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:53:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.044883636, 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.044681399, 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:53:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:53:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:53:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.044491588, 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:53:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:53:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.044179585, 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:53:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.043937098000001, 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:53:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:53:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.043778136, 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:53:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.043143865999999, 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:53:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.042929755999999, 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:53:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:53:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:53:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.042793286, 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:53:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:53:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.042540806, 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:53:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:49.661 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:49.663 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:49.664 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:49.661+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:49.665 INFO:teuthology.orchestra.run.smithi076.stdout:1665 writesame 0xe5274de thru 0xe53749e data_size 0x1(0xffc1 bytes) 2024-09-14T22:53:49.665 INFO:teuthology.orchestra.run.smithi076.stdout:1666 writesame 0x1a4174 thru 0x1a4aa2 data_size 0x1(0x92f bytes) 2024-09-14T22:53:49.665 INFO:teuthology.orchestra.run.smithi076.stdout:1667 read 0x79e12f2 thru 0x79f119e (0xfead bytes) 2024-09-14T22:53:49.665 INFO:teuthology.orchestra.run.smithi076.stdout:1668 write 0xa4533ef thru 0xa453547 (0x159 bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1669 write 0x2bb0b16 thru 0x2bb6153 (0x563e bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1672 read 0x9477130 thru 0x947f439 (0x830a bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1675 write 0x9710730 thru 0x9714ef2 (0x47c3 bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1676 read 0xd5af8b6 thru 0xd5b97b7 (0x9f02 bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1677 write 0xd8fce66 thru 0xd9044a2 (0x763d bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1682 read 0x20be0a5 thru 0x20c9f4e (0xbeaa bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1683 read 0xc97268b thru 0xc975d40 (0x36b6 bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1684 write 0x4b41323 thru 0x4b4d341 (0xc01f bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1685 write 0xe90c201 thru 0xe918bfa (0xc9fa bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1689 punch from 0x2f387dc to 0x2f435fa, (0xae1e bytes) 2024-09-14T22:53:49.666 INFO:teuthology.orchestra.run.smithi076.stdout:1691 writesame 0xb663b1a thru 0xb66bdc4 data_size 0x1(0x82ab bytes) 2024-09-14T22:53:49.667 INFO:teuthology.orchestra.run.smithi076.stdout:1692 writesame 0x2ec98ac thru 0x2ed3fed data_size 0x4f(0xa742 bytes) 2024-09-14T22:53:49.667 INFO:teuthology.orchestra.run.smithi076.stdout:1695 write 0x12cf9d4 thru 0x12d9c24 (0xa251 bytes) 2024-09-14T22:53:49.667 INFO:teuthology.orchestra.run.smithi076.stdout:1696 read 0x788adea thru 0x789aa41 (0xfc58 bytes) 2024-09-14T22:53:49.667 INFO:teuthology.orchestra.run.smithi076.stdout:1697 read 0xbcafe28 thru 0xbcb1448 (0x1621 bytes) 2024-09-14T22:53:49.667 INFO:teuthology.orchestra.run.smithi076.stdout:1698 writesame 0x26842d2 thru 0x2684d46 data_size 0x1(0xa75 bytes) 2024-09-14T22:53:49.667 INFO:teuthology.orchestra.run.smithi076.stdout:1699 trunc from 0xe918bfb to 0x69795f4 2024-09-14T22:53:49.667 INFO:teuthology.orchestra.run.smithi076.stdout:1700 writesame 0x735cf2a thru 0x736a7ec data_size 0x35(0xd8c3 bytes) 2024-09-14T22:53:49.667 INFO:teuthology.orchestra.run.smithi076.stdout:1702 writesame 0x18a8abf thru 0x18aee17 data_size 0x1(0x6359 bytes) 2024-09-14T22:53:49.678 INFO:teuthology.orchestra.run.smithi076.stdout:1703 writesame 0x4752b31 thru 0x4761bbe data_size 0x29(0xf08e bytes) 2024-09-14T22:53:49.680 INFO:teuthology.orchestra.run.smithi076.stdout:1704 punch from 0x2cf1de5 to 0x2cf3b56, (0x1d71 bytes) 2024-09-14T22:53:49.681 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.681 INFO:teuthology.orchestra.run.smithi076.stdout:1706 punch from 0x353ffed to 0x354d97c, (0xd98f bytes) 2024-09-14T22:53:49.681 INFO:teuthology.orchestra.run.smithi076.stdout:1707 punch from 0x6872724 to 0x687a4ec, (0x7dc8 bytes) 2024-09-14T22:53:49.681 INFO:teuthology.orchestra.run.smithi076.stdout:1709 read 0xc4a457 thru 0xc4dafe (0x36a8 bytes) 2024-09-14T22:53:49.681 INFO:teuthology.orchestra.run.smithi076.stdout:1711 read 0x232999f thru 0x2336cd6 (0xd338 bytes) 2024-09-14T22:53:49.693 INFO:teuthology.orchestra.run.smithi076.stdout:1712 writesame 0xcec4d37 thru 0xced22ea data_size 0x5e(0xd5b4 bytes) 2024-09-14T22:53:49.693 INFO:teuthology.orchestra.run.smithi076.stdout:1713 write 0x731d233 thru 0x7326839 (0x9607 bytes) 2024-09-14T22:53:49.703 INFO:teuthology.orchestra.run.smithi076.stdout:1714 write 0xdf362fc thru 0xdf40b90 (0xa895 bytes) 2024-09-14T22:53:49.711 INFO:teuthology.orchestra.run.smithi076.stdout:1715 writesame 0xe2289c1 thru 0xe237c88 data_size 0x44(0xf2c8 bytes) 2024-09-14T22:53:49.711 INFO:teuthology.orchestra.run.smithi076.stdout:1716 write 0x7d68c7d thru 0x7d6ccbd (0x4041 bytes) 2024-09-14T22:53:49.712 INFO:teuthology.orchestra.run.smithi076.stdout:1717 write 0x2782af thru 0x28536b (0xd0bd bytes) 2024-09-14T22:53:49.713 INFO:teuthology.orchestra.run.smithi076.stdout:1719 writesame 0x418d288 thru 0x418ea10 data_size 0x5(0x1789 bytes) 2024-09-14T22:53:49.715 INFO:teuthology.orchestra.run.smithi076.stdout:1721 read 0x86d07ec thru 0x86de6d5 (0xdeea bytes) 2024-09-14T22:53:49.715 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.715 INFO:teuthology.orchestra.run.smithi076.stdout:1723 read 0x3a0f3f7 thru 0x3a10c2e (0x1838 bytes) 2024-09-14T22:53:49.715 INFO:teuthology.orchestra.run.smithi076.stdout:1724 read 0x1365d7d thru 0x1374757 (0xe9db bytes) 2024-09-14T22:53:49.716 INFO:teuthology.orchestra.run.smithi076.stdout:1725 read 0xa274ad3 thru 0xa27d5ba (0x8ae8 bytes) 2024-09-14T22:53:49.716 INFO:teuthology.orchestra.run.smithi076.stdout:1726 read 0x27d2374 thru 0x27de7e2 (0xc46f bytes) 2024-09-14T22:53:49.716 INFO:teuthology.orchestra.run.smithi076.stdout:1728 write 0xd24beaa thru 0xd24f7c7 (0x391e bytes) 2024-09-14T22:53:49.720 INFO:teuthology.orchestra.run.smithi076.stdout:1729 read 0xa531416 thru 0xa536131 (0x4d1c bytes) 2024-09-14T22:53:49.720 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.720 INFO:teuthology.orchestra.run.smithi076.stdout:1731 read 0x19bbee6 thru 0x19c0cdc (0x4df7 bytes) 2024-09-14T22:53:49.720 INFO:teuthology.orchestra.run.smithi076.stdout:1732 writesame 0x8a6f795 thru 0x8a6fc66 data_size 0x1(0x4d2 bytes) 2024-09-14T22:53:49.724 INFO:teuthology.orchestra.run.smithi076.stdout:1733 punch from 0x89f16ae to 0x89fbebe, (0xa810 bytes) 2024-09-14T22:53:49.724 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.724 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.730 INFO:teuthology.orchestra.run.smithi076.stdout:1737 writesame 0xe9b6e11 thru 0xe9c6a2f data_size 0x2b(0xfc1f bytes) 2024-09-14T22:53:49.730 INFO:teuthology.orchestra.run.smithi076.stdout:1738 writesame 0xcad52a5 thru 0xcada9b9 data_size 0x3(0x5715 bytes) 2024-09-14T22:53:49.735 INFO:teuthology.orchestra.run.smithi076.stdout:1739 write 0x57669ae thru 0x576897e (0x1fd1 bytes) 2024-09-14T22:53:49.737 INFO:teuthology.orchestra.run.smithi076.stdout:1740 write 0xb873fc9 thru 0xb87556d (0x15a5 bytes) 2024-09-14T22:53:49.740 INFO:teuthology.orchestra.run.smithi076.stdout:1741 writesame 0xbf45fc5 thru 0xbf466df data_size 0x1(0x71b bytes) 2024-09-14T22:53:49.742 INFO:teuthology.orchestra.run.smithi076.stdout:1742 write 0x85efe26 thru 0x85f3cf6 (0x3ed1 bytes) 2024-09-14T22:53:49.746 INFO:teuthology.orchestra.run.smithi076.stdout:1743 write 0x957dce8 thru 0x95868a5 (0x8bbe bytes) 2024-09-14T22:53:49.747 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:49.747 INFO:teuthology.orchestra.run.smithi076.stdout:1745 punch from 0x96f5970 to 0x9704a20, (0xf0b0 bytes) 2024-09-14T22:53:49.747 INFO:teuthology.orchestra.run.smithi076.stdout:1746 write 0xa8352e9 thru 0xa83d00e (0x7d26 bytes) 2024-09-14T22:53:49.749 INFO:teuthology.orchestra.run.smithi076.stdout:1747 read 0x6e5f157 thru 0x6e623a6 (0x3250 bytes) 2024-09-14T22:53:49.749 INFO:teuthology.orchestra.run.smithi076.stdout:1749 write 0x73b9e1b thru 0x73ba4f2 (0x6d8 bytes) 2024-09-14T22:53:49.750 INFO:teuthology.orchestra.run.smithi076.stdout:1750 punch from 0xce4827d to 0xce5045c, (0x81df bytes) 2024-09-14T22:53:49.750 INFO:teuthology.orchestra.run.smithi076.stdout:1751 write 0xbc92c72 thru 0xbc9e0de (0xb46d bytes) 2024-09-14T22:53:49.751 INFO:teuthology.orchestra.run.smithi076.stdout:1752 write 0x903b14b thru 0x903ca0a (0x18c0 bytes) 2024-09-14T22:53:49.753 INFO:teuthology.orchestra.run.smithi076.stdout:1753 clone 18 order 21 su 65536 sc 4 2024-09-14T22:53:49.763 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:49.761+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:49.810 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:49.810 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:49.811 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:49.819 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:49.903 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:49.902+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:49.921 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:49.921 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:49.921 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T22:53:49.921 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.068412 2024-09-14T22:53:49.921 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.0684120119314151 2024-09-14T22:53:49.965 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:49.964+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:49.966 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:49.966 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:49.966 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:49.966 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:49.966 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:49.976 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:50.013 INFO:teuthology.orchestra.run.smithi076.stdout:flattening image image_client.0-clone17 2024-09-14T22:53:50.066 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:50.064+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.036843617999999, 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:53:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.036625722, 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:53:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.033942075999999, 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:53:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.033783661000001, 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:53:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:53:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.033756395000001, 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:53:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:53:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.033549254, 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:53:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.03351499, 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:53:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:53:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.031882943999999, 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:53:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.031858331999999, 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:53:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:53:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.031724046000001, 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:53:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:53:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:53:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.030674338000001, 2024-09-14T22:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:53:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:53:50.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.030572153000001, 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:53:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:53:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.030548248999999, 2024-09-14T22:53:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:53:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.030514729, 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:53:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.030360083000001, 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:53:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.030318908999998, 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:53:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:53:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.030290748999999, 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:53:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.030101358, 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:53:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:53:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.114011530999999, 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.1139917, 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.213 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:50.215 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:50.215 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:50.212+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:50.216 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T22:53:50.216 INFO:teuthology.orchestra.run.smithi076.stdout:1755 read 0x458d203 thru 0x459c021 (0xee1f bytes) 2024-09-14T22:53:50.216 INFO:teuthology.orchestra.run.smithi076.stdout:1756 read 0xaf5954d thru 0xaf5d9f5 (0x44a9 bytes) 2024-09-14T22:53:50.216 INFO:teuthology.orchestra.run.smithi076.stdout:1757 write 0x98bb4b4 thru 0x98c33dd (0x7f2a bytes) 2024-09-14T22:53:50.216 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.216 INFO:teuthology.orchestra.run.smithi076.stdout:1760 read 0x36f0eb thru 0x376f23 (0x7e39 bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1761 read 0xbf82ad0 thru 0xbf8648c (0x39bd bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1764 read 0xcdb0110 thru 0xcdb85bc (0x84ad bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1766 read 0x4b570c thru 0x4b9375 (0x3c6a bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1769 read 0xcf169f5 thru 0xcf16e99 (0x4a5 bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1773 write 0x38b37b5 thru 0x38bf5a4 (0xbdf0 bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1774 writesame 0x4354dcb thru 0x4360df2 data_size 0x58(0xc028 bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1777 read 0x96640d0 thru 0x967009e (0xbfcf bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1781 write 0x550c8df thru 0x55185c4 (0xbce6 bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1782 write 0x7872512 thru 0x787e3b3 (0xbea2 bytes) 2024-09-14T22:53:50.217 INFO:teuthology.orchestra.run.smithi076.stdout:1783 punch from 0x3f86c12 to 0x3f8b80b, (0x4bf9 bytes) 2024-09-14T22:53:50.218 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.218 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.218 INFO:teuthology.orchestra.run.smithi076.stdout:1786 write 0x8ffd17b thru 0x9000587 (0x340d bytes) 2024-09-14T22:53:50.220 INFO:teuthology.orchestra.run.smithi076.stdout:1787 read 0xb18cfec thru 0xb190ad1 (0x3ae6 bytes) 2024-09-14T22:53:50.220 INFO:teuthology.orchestra.run.smithi076.stdout:1788 punch from 0x71ac938 to 0x71b9baa, (0xd272 bytes) 2024-09-14T22:53:50.220 INFO:teuthology.orchestra.run.smithi076.stdout:1789 write 0x104c0b thru 0x10f53e (0xa934 bytes) 2024-09-14T22:53:50.228 INFO:teuthology.orchestra.run.smithi076.stdout:1790 read 0x99e3a9b thru 0x99edd05 (0xa26b bytes) 2024-09-14T22:53:50.228 INFO:teuthology.orchestra.run.smithi076.stdout:1791 read 0x12b362d thru 0x12b384e (0x222 bytes) 2024-09-14T22:53:50.229 INFO:teuthology.orchestra.run.smithi076.stdout:1792 punch from 0xc081d6 to 0xc0fc0d, (0x7a37 bytes) 2024-09-14T22:53:50.229 INFO:teuthology.orchestra.run.smithi076.stdout:1793 writesame 0x45db965 thru 0x45ea9c1 data_size 0x81(0xf05d bytes) 2024-09-14T22:53:50.231 INFO:teuthology.orchestra.run.smithi076.stdout:1794 punch from 0x2c60bd3 to 0x2c62d40, (0x216d bytes) 2024-09-14T22:53:50.231 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.231 INFO:teuthology.orchestra.run.smithi076.stdout:1796 write 0x3f5961 thru 0x403631 (0xdcd1 bytes) 2024-09-14T22:53:50.239 INFO:teuthology.orchestra.run.smithi076.stdout:1797 read 0x55d86ed thru 0x55dd752 (0x5066 bytes) 2024-09-14T22:53:50.239 INFO:teuthology.orchestra.run.smithi076.stdout:1799 read 0x9543ba6 thru 0x954bbfb (0x8056 bytes) 2024-09-14T22:53:50.249 INFO:teuthology.orchestra.run.smithi076.stdout:1800 trunc from 0xe9c6a30 to 0x7f72501 2024-09-14T22:53:50.272 INFO:teuthology.orchestra.run.smithi076.stdout:1801 trunc from 0x7f72501 to 0xb9fb163 2024-09-14T22:53:50.279 INFO:teuthology.orchestra.run.smithi076.stdout:1802 read 0x204b942 thru 0x20575cf (0xbc8e bytes) 2024-09-14T22:53:50.279 INFO:teuthology.orchestra.run.smithi076.stdout:1803 writesame 0x69704b3 thru 0x6976e4f data_size 0x13(0x699d bytes) 2024-09-14T22:53:50.285 INFO:teuthology.orchestra.run.smithi076.stdout:1804 read 0x7f033d1 thru 0x7f0f400 (0xc030 bytes) 2024-09-14T22:53:50.285 INFO:teuthology.orchestra.run.smithi076.stdout:1805 write 0x3fbb321 thru 0x3fc1ce8 (0x69c8 bytes) 2024-09-14T22:53:50.301 INFO:teuthology.orchestra.run.smithi076.stdout:1807 write 0xdf5ee06 thru 0xdf66dce (0x7fc9 bytes) 2024-09-14T22:53:50.306 INFO:teuthology.orchestra.run.smithi076.stdout:1809 write 0x8ec52ba thru 0x8ecc681 (0x73c8 bytes) 2024-09-14T22:53:50.309 INFO:teuthology.orchestra.run.smithi076.stdout:1811 read 0x2b33b6c thru 0x2b37599 (0x3a2e bytes) 2024-09-14T22:53:50.310 INFO:teuthology.orchestra.run.smithi076.stdout:1812 read 0xda2796d thru 0xda2ed58 (0x73ec bytes) 2024-09-14T22:53:50.311 INFO:teuthology.orchestra.run.smithi076.stdout:1813 write 0xc0a3e87 thru 0xc0a8ca3 (0x4e1d bytes) 2024-09-14T22:53:50.314 INFO:teuthology.orchestra.run.smithi076.stdout:1814 punch from 0x876e633 to 0x877a8ad, (0xc27a bytes) 2024-09-14T22:53:50.314 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:50.313+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:50.318 INFO:teuthology.orchestra.run.smithi076.stdout:1815 write 0xe394819 thru 0xe39b094 (0x687c bytes) 2024-09-14T22:53:50.321 INFO:teuthology.orchestra.run.smithi076.stdout:1816 write 0x1f861eb thru 0x1f8bdb9 (0x5bcf bytes) 2024-09-14T22:53:50.327 INFO:teuthology.orchestra.run.smithi076.stdout:1817 write 0x7bc5e72 thru 0x7bd1c31 (0xbdc0 bytes) 2024-09-14T22:53:50.329 INFO:teuthology.orchestra.run.smithi076.stdout:1819 punch from 0x8a9d96 to 0x8ab8a8, (0x1b12 bytes) 2024-09-14T22:53:50.329 INFO:teuthology.orchestra.run.smithi076.stdout:1821 read 0xb8af235 thru 0xb8bb009 (0xbdd5 bytes) 2024-09-14T22:53:50.329 INFO:teuthology.orchestra.run.smithi076.stdout:1822 writesame 0x690c900 thru 0x691853c data_size 0x1(0xbc3d bytes) 2024-09-14T22:53:50.358 INFO:teuthology.orchestra.run.smithi076.stdout:1823 trunc from 0xe39b095 to 0xc119d12 2024-09-14T22:53:50.367 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:50.368 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:50.368 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:50.368 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:50.372 INFO:teuthology.orchestra.run.smithi076.stdout:1824 read 0x6e5b146 thru 0x6e6066a (0x5525 bytes) 2024-09-14T22:53:50.372 INFO:teuthology.orchestra.run.smithi076.stdout:1825 write 0x3e406e6 thru 0x3e4e26b (0xdb86 bytes) 2024-09-14T22:53:50.374 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.374 INFO:teuthology.orchestra.run.smithi076.stdout:1827 write 0xab57b30 thru 0xab64b7e (0xd04f bytes) 2024-09-14T22:53:50.377 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:50.378 INFO:teuthology.orchestra.run.smithi076.stdout:1828 read 0x9785ceb thru 0x978e184 (0x849a bytes) 2024-09-14T22:53:50.379 INFO:teuthology.orchestra.run.smithi076.stdout:1829 write 0x8a49743 thru 0x8a49aa2 (0x360 bytes) 2024-09-14T22:53:50.380 INFO:teuthology.orchestra.run.smithi076.stdout:1830 punch from 0x2a985dc to 0x2a98c60, (0x684 bytes) 2024-09-14T22:53:50.380 INFO:teuthology.orchestra.run.smithi076.stdout:1832 writesame 0xbdc8f95 thru 0xbdd1987 data_size 0x7(0x89f3 bytes) 2024-09-14T22:53:50.385 INFO:teuthology.orchestra.run.smithi076.stdout:1833 read 0x8e48907 thru 0x8e4d655 (0x4d4f bytes) 2024-09-14T22:53:50.388 INFO:teuthology.orchestra.run.smithi076.stdout:1834 read 0xae4cc5 thru 0xaea316 (0x5652 bytes) 2024-09-14T22:53:50.388 INFO:teuthology.orchestra.run.smithi076.stdout:1835 write 0x447647c thru 0x4477429 (0xfae bytes) 2024-09-14T22:53:50.390 INFO:teuthology.orchestra.run.smithi076.stdout:1836 writesame 0x3b94a10 thru 0x3b959bc data_size 0x1(0xfad bytes) 2024-09-14T22:53:50.396 INFO:teuthology.orchestra.run.smithi076.stdout:1838 write 0x21fe65f thru 0x220369b (0x503d bytes) 2024-09-14T22:53:50.404 INFO:teuthology.orchestra.run.smithi076.stdout:1840 write 0xe618ba thru 0xe6ad07 (0x944e bytes) 2024-09-14T22:53:50.413 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.413 INFO:teuthology.orchestra.run.smithi076.stdout:1842 read 0x83580e1 thru 0x835e6f9 (0x6619 bytes) 2024-09-14T22:53:50.414 INFO:teuthology.orchestra.run.smithi076.stdout:1843 flatten 2024-09-14T22:53:50.414 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:50.413+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:50.497 INFO:teuthology.orchestra.run.smithi076.stdout:1844 read 0x185c84f thru 0x18695b2 (0xcd64 bytes) 2024-09-14T22:53:50.498 INFO:teuthology.orchestra.run.smithi076.stdout:1845 read 0x8536577 thru 0x853d497 (0x6f21 bytes) 2024-09-14T22:53:50.498 INFO:teuthology.orchestra.run.smithi076.stdout:1846 read 0x4158b0f thru 0x415c674 (0x3b66 bytes) 2024-09-14T22:53:50.498 INFO:teuthology.orchestra.run.smithi076.stdout:1848 writesame 0x8f0139f thru 0x8f0baaa data_size 0x2(0xa70c bytes) 2024-09-14T22:53:50.511 INFO:teuthology.orchestra.run.smithi076.stdout:1849 writesame 0x1aa671 thru 0x1b05ea data_size 0x65(0x5f7a bytes) 2024-09-14T22:53:50.513 INFO:teuthology.orchestra.run.smithi076.stdout:1850 punch from 0x91c4213 to 0x91c7e64, (0x3c51 bytes) 2024-09-14T22:53:50.513 INFO:teuthology.orchestra.run.smithi076.stdout:1851 read 0x33d10a8 thru 0x33d2e44 (0x1d9d bytes) 2024-09-14T22:53:50.515 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:50.514+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:50.517 INFO:teuthology.orchestra.run.smithi076.stdout:1854 trunc from 0xc119d12 to 0x8b29671 2024-09-14T22:53:50.529 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.529 INFO:teuthology.orchestra.run.smithi076.stdout:1858 read 0x6ed23b1 thru 0x6ed343c (0x108c bytes) 2024-09-14T22:53:50.529 INFO:teuthology.orchestra.run.smithi076.stdout:1860 read 0x40171f thru 0x40645a (0x4d3c bytes) 2024-09-14T22:53:50.530 INFO:teuthology.orchestra.run.smithi076.stdout:1861 write 0x86283c9 thru 0x8632b01 (0xa739 bytes) 2024-09-14T22:53:50.533 INFO:teuthology.orchestra.run.smithi076.stdout:1864 read 0x87b10df thru 0x87bf67d (0xe59f bytes) 2024-09-14T22:53:50.534 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.534 INFO:teuthology.orchestra.run.smithi076.stdout:1866 read 0x3303578 thru 0x3312e01 (0xf88a bytes) 2024-09-14T22:53:50.534 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.534 INFO:teuthology.orchestra.run.smithi076.stdout:1868 read 0xb8f750 thru 0xb9c0af (0xc960 bytes) 2024-09-14T22:53:50.534 INFO:teuthology.orchestra.run.smithi076.stdout:1871 read 0x3e2c854 thru 0x3e35aa0 (0x924d bytes) 2024-09-14T22:53:50.535 INFO:teuthology.orchestra.run.smithi076.stdout:1872 read 0x33f8cf8 thru 0x3406fd1 (0xe2da bytes) 2024-09-14T22:53:50.535 INFO:teuthology.orchestra.run.smithi076.stdout:1873 read 0x486ee96 thru 0x48766ee (0x7859 bytes) 2024-09-14T22:53:50.547 INFO:teuthology.orchestra.run.smithi076.stdout:1874 writesame 0xe649afd thru 0xe657d9b data_size 0x29(0xe29f bytes) 2024-09-14T22:53:50.549 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:50.549 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:50.549 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:50.549 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:50.549 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:50.557 INFO:teuthology.orchestra.run.smithi076.stdout:1875 trunc from 0xe657d9c to 0x5c02cc5 2024-09-14T22:53:50.558 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:50.567 INFO:teuthology.orchestra.run.smithi076.stdout:1876 writesame 0xb88d81 thru 0xb8dfbc data_size 0x26(0x523c bytes) 2024-09-14T22:53:50.569 INFO:teuthology.orchestra.run.smithi076.stdout:1878 read 0x2a42295 thru 0x2a42458 (0x1c4 bytes) 2024-09-14T22:53:50.569 INFO:teuthology.orchestra.run.smithi076.stdout:1881 read 0x43257be thru 0x432d016 (0x7859 bytes) 2024-09-14T22:53:50.578 INFO:teuthology.orchestra.run.smithi076.stdout:1882 write 0x87b3ae6 thru 0x87bea49 (0xaf64 bytes) 2024-09-14T22:53:50.581 INFO:teuthology.orchestra.run.smithi076.stdout:1883 punch from 0x8234d to 0x834f8, (0x11ab bytes) 2024-09-14T22:53:50.581 INFO:teuthology.orchestra.run.smithi076.stdout:1885 write 0x7d2bebb thru 0x7d31900 (0x5a46 bytes) 2024-09-14T22:53:50.590 INFO:teuthology.orchestra.run.smithi076.stdout:1886 trunc from 0x87bea4a to 0x3b8cdc6 2024-09-14T22:53:50.613 INFO:teuthology.orchestra.run.smithi076.stdout:1887 trunc from 0x3b8cdc6 to 0xe50886d 2024-09-14T22:53:50.618 INFO:teuthology.orchestra.run.smithi076.stdout:1894 read 0xc8043a5 thru 0xc80e232 (0x9e8e bytes) 2024-09-14T22:53:50.619 INFO:teuthology.orchestra.run.smithi076.stdout:1896 writesame 0x8e9c4a1 thru 0x8ea1810 data_size 0x59(0x5370 bytes) 2024-09-14T22:53:50.640 INFO:teuthology.orchestra.run.smithi076.stdout:1897 trunc from 0xe50886d to 0xe9b9b1 2024-09-14T22:53:50.643 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:50.642+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:50.654 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.654 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.654 INFO:teuthology.orchestra.run.smithi076.stdout:1900 read 0x77b3ce thru 0x788f56 (0xdb89 bytes) 2024-09-14T22:53:50.664 INFO:teuthology.orchestra.run.smithi076.stdout:1901 write 0x50262d5 thru 0x5026e2e (0xb5a bytes) 2024-09-14T22:53:50.674 INFO:teuthology.orchestra.run.smithi076.stdout:1903 write 0x7244656 thru 0x7247dab (0x3756 bytes) 2024-09-14T22:53:50.676 INFO:teuthology.orchestra.run.smithi076.stdout:1904 read 0x4c2ade thru 0x4c5b33 (0x3056 bytes) 2024-09-14T22:53:50.691 INFO:teuthology.orchestra.run.smithi076.stdout:1906 writesame 0xde3dd63 thru 0xde44bf5 data_size 0x1(0x6e93 bytes) 2024-09-14T22:53:50.691 INFO:teuthology.orchestra.run.smithi076.stdout:1907 read 0x5342734 thru 0x534df61 (0xb82e bytes) 2024-09-14T22:53:50.691 INFO:teuthology.orchestra.run.smithi076.stdout:1909 read 0x9e3d1f5 thru 0x9e436a0 (0x64ac bytes) 2024-09-14T22:53:50.691 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.691 INFO:teuthology.orchestra.run.smithi076.stdout:1912 write 0x2f89e8d thru 0x2f8a64b (0x7bf bytes) 2024-09-14T22:53:50.693 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.693 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.693 INFO:teuthology.orchestra.run.smithi076.stdout:1915 write 0x9e2f696 thru 0x9e3b70a (0xc075 bytes) 2024-09-14T22:53:50.701 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.705 INFO:teuthology.orchestra.run.smithi076.stdout:1918 trunc from 0xde44bf6 to 0xacb7b03 2024-09-14T22:53:50.707 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:50.707 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.197810955000001, 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.708 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:50.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.333153100000001, 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.715 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.719 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.333047164, 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:53:50.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:53:50.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332972463000001, 2024-09-14T22:53:50.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:53:50.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332886982, 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.729 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:53:50.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:53:50.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332740596000001, 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:53:50.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:53:50.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332690596000001, 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.739 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:53:50.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:53:50.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332527431999999, 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332489122, 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:53:50.748 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:53:50.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332355453, 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:53:50.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.757 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332274886, 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:53:50.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.762 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332177949, 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:53:50.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.332126393999999, 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:53:50.767 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:53:50.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:53:50.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.329001764999999, 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.772 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:53:50.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:53:50.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:53:50.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.328870152, 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.777 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:53:50.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:53:50.781 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.328583739999999, 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:53:50.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:53:50.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118419 3.15 3.2a32cf5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377105+0000", 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.328481975000001, 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050052747000000002, 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.786 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118419 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377105+0000", 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377105+0000", 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377105+0000", 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8100000000000001e-07 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377110+0000", 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3200000000000001e-06 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377121+0000", 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1131999999999999e-05 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377128+0000", 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2760000000000003e-06 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427153+0000", 2024-09-14T22:53:50.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050024787000000001 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427157+0000", 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3510000000000004e-06 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118420 3.15 3.2a32cf5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377140+0000", 2024-09-14T22:53:50.790 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.328446895999999, 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050044040999999997, 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118420 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377140+0000", 2024-09-14T22:53:50.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377140+0000", 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377140+0000", 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377143+0000", 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7159999999999999e-06 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377150+0000", 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6950000000000003e-06 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377155+0000", 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2360000000000004e-06 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427176+0000", 2024-09-14T22:53:50.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050020792000000001 2024-09-14T22:53:50.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427184+0000", 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1100000000000003e-06 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118421 3.15 3.2a32cf5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377162+0000", 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.328424782000001, 2024-09-14T22:53:50.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050048141999999997, 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118421 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377162+0000", 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377162+0000", 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377162+0000", 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377165+0000", 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2670000000000001e-06 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377171+0000", 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1750000000000002e-06 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377187+0000", 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6337e-05 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427203+0000", 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050016400000000003 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427210+0000", 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5139999999999998e-06 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.800 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118432 3.15 3.2a32cf5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377584+0000", 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.328002936000001, 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050039961000000001, 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118432 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377584+0000", 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377584+0000", 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377689+0000", 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010526699999999999 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377693+0000", 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.681e-06 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377701+0000", 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9359999999999999e-06 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377707+0000", 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.998e-06 2024-09-14T22:53:50.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427618+0000", 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049911075999999999 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427624+0000", 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.003e-06 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.804 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:50.805 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:50.805 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:50.807 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:50.805+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:50.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:50.808 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.808 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.808 INFO:teuthology.orchestra.run.smithi076.stdout:1922 write 0x2418d9b thru 0x241ae3f (0x20a5 bytes) 2024-09-14T22:53:50.808 INFO:teuthology.orchestra.run.smithi076.stdout:1923 write 0xec0dc17 thru 0xec1872e (0xab18 bytes) 2024-09-14T22:53:50.808 INFO:teuthology.orchestra.run.smithi076.stdout:1926 punch from 0xb40f469 to 0xb41ca41, (0xd5d8 bytes) 2024-09-14T22:53:50.808 INFO:teuthology.orchestra.run.smithi076.stdout:1927 writesame 0xe5b12d1 thru 0xe5c071d data_size 0x3(0xf44d bytes) 2024-09-14T22:53:50.808 INFO:teuthology.orchestra.run.smithi076.stdout:1928 write 0x72354af thru 0x723b728 (0x627a bytes) 2024-09-14T22:53:50.808 INFO:teuthology.orchestra.run.smithi076.stdout:1929 write 0x3d59ef8 thru 0x3d61913 (0x7a1c bytes) 2024-09-14T22:53:50.809 INFO:teuthology.orchestra.run.smithi076.stdout:1930 writesame 0x18cced0 thru 0x18cf63d data_size 0x31(0x276e bytes) 2024-09-14T22:53:50.809 INFO:teuthology.orchestra.run.smithi076.stdout:1931 write 0xb8fa0c6 thru 0xb8fc34d (0x2288 bytes) 2024-09-14T22:53:50.809 INFO:teuthology.orchestra.run.smithi076.stdout:1934 write 0x19896e6 thru 0x1990849 (0x7164 bytes) 2024-09-14T22:53:50.809 INFO:teuthology.orchestra.run.smithi076.stdout:1935 trunc from 0xec1872f to 0xd986 2024-09-14T22:53:50.809 INFO:teuthology.orchestra.run.smithi076.stdout:1937 write 0x26347b6 thru 0x26402b9 (0xbb04 bytes) 2024-09-14T22:53:50.813 INFO:teuthology.orchestra.run.smithi076.stdout:1938 read 0x1761574 thru 0x176cbe7 (0xb674 bytes) 2024-09-14T22:53:50.813 INFO:teuthology.orchestra.run.smithi076.stdout:1939 read 0x19006ca thru 0x1903edf (0x3816 bytes) 2024-09-14T22:53:50.813 INFO:teuthology.orchestra.run.smithi076.stdout:1940 read 0x455db8 thru 0x45730d (0x1556 bytes) 2024-09-14T22:53:50.814 INFO:teuthology.orchestra.run.smithi076.stdout:1941 trunc from 0x26402ba to 0x176bf80 2024-09-14T22:53:50.837 INFO:teuthology.orchestra.run.smithi076.stdout:1942 write 0x6a7e0a8 thru 0x6a83780 (0x56d9 bytes) 2024-09-14T22:53:50.842 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.842 INFO:teuthology.orchestra.run.smithi076.stdout:1945 read 0x4f8d041 thru 0x4f912ae (0x426e bytes) 2024-09-14T22:53:50.842 INFO:teuthology.orchestra.run.smithi076.stdout:1947 punch from 0x6853669 to 0x6859c38, (0x65cf bytes) 2024-09-14T22:53:50.842 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.842 INFO:teuthology.orchestra.run.smithi076.stdout:1949 punch from 0x23399db to 0x234932b, (0xf950 bytes) 2024-09-14T22:53:50.843 INFO:teuthology.orchestra.run.smithi076.stdout:1950 trunc from 0x6a83781 to 0x79fb5dc 2024-09-14T22:53:50.849 INFO:teuthology.orchestra.run.smithi076.stdout:1951 trunc from 0x79fb5dc to 0x8a5db45 2024-09-14T22:53:50.855 INFO:teuthology.orchestra.run.smithi076.stdout:1952 read 0x262fd1f thru 0x2630283 (0x565 bytes) 2024-09-14T22:53:50.855 INFO:teuthology.orchestra.run.smithi076.stdout:1953 writesame 0x88dda4b thru 0x88e0285 data_size 0x1(0x283b bytes) 2024-09-14T22:53:50.863 INFO:teuthology.orchestra.run.smithi076.stdout:1956 read 0x13b3702 thru 0x13c2b01 (0xf400 bytes) 2024-09-14T22:53:50.864 INFO:teuthology.orchestra.run.smithi076.stdout:1958 writesame 0x34ce782 thru 0x34d5b14 data_size 0x1(0x7393 bytes) 2024-09-14T22:53:50.882 INFO:teuthology.orchestra.run.smithi076.stdout:1959 read 0x22a59c8 thru 0x22ac2dc (0x6915 bytes) 2024-09-14T22:53:50.882 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.882 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.891 INFO:teuthology.orchestra.run.smithi076.stdout:1962 write 0xabb7b4c thru 0xabc251d (0xa9d2 bytes) 2024-09-14T22:53:50.896 INFO:teuthology.orchestra.run.smithi076.stdout:1967 read 0x13c1a31 thru 0x13c6c2f (0x51ff bytes) 2024-09-14T22:53:50.896 INFO:teuthology.orchestra.run.smithi076.stdout:1968 punch from 0x76b3bfd to 0x76bf12a, (0xb52d bytes) 2024-09-14T22:53:50.907 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:50.905+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:50.908 INFO:teuthology.orchestra.run.smithi076.stdout:1971 trunc from 0xabc251e to 0xe8ddc0 2024-09-14T22:53:50.926 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:50.927 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:50.927 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:50.927 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:50.929 INFO:teuthology.orchestra.run.smithi076.stdout:1972 trunc from 0xe8ddc0 to 0x9bc24e9 2024-09-14T22:53:50.935 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:53:50.935 INFO:teuthology.orchestra.run.smithi076.stdout:1974 write 0x3928680 thru 0x393297e (0xa2ff bytes) 2024-09-14T22:53:50.938 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:50.940 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.1 primary-affinity to 0.068412 (844832) 2024-09-14T22:53:50.942 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.943 INFO:teuthology.orchestra.run.smithi076.stdout:1976 trunc from 0x9bc24e9 to 0x83b51b8 2024-09-14T22:53:50.957 INFO:teuthology.orchestra.run.smithi076.stdout:1977 trunc from 0x83b51b8 to 0xd96d5cf 2024-09-14T22:53:50.962 INFO:teuthology.orchestra.run.smithi076.stdout:1979 write 0xc79455c thru 0xc796ca1 (0x2746 bytes) 2024-09-14T22:53:50.966 INFO:teuthology.orchestra.run.smithi076.stdout:1980 punch from 0x116086d to 0x116956a, (0x8cfd bytes) 2024-09-14T22:53:50.966 INFO:teuthology.orchestra.run.smithi076.stdout:1981 read 0x439d95f thru 0x43aa7dd (0xce7f bytes) 2024-09-14T22:53:50.966 INFO:teuthology.orchestra.run.smithi076.stdout:1982 read 0x13f2dd6 thru 0x13f37bb (0x9e6 bytes) 2024-09-14T22:53:50.966 INFO:teuthology.orchestra.run.smithi076.stdout:1983 read 0x270b1b thru 0x2755ff (0x4ae5 bytes) 2024-09-14T22:53:50.966 INFO:teuthology.orchestra.run.smithi076.stdout:1985 write 0x754405 thru 0x7569ca (0x25c6 bytes) 2024-09-14T22:53:50.969 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.969 INFO:teuthology.orchestra.run.smithi076.stdout:1988 read 0x9f30f70 thru 0x9f353af (0x4440 bytes) 2024-09-14T22:53:50.969 INFO:teuthology.orchestra.run.smithi076.stdout:1991 read 0xa088aeb thru 0xa09052e (0x7a44 bytes) 2024-09-14T22:53:50.969 INFO:teuthology.orchestra.run.smithi076.stdout:1993 punch from 0x5ddc241 to 0x5de879e, (0xc55d bytes) 2024-09-14T22:53:50.969 INFO:teuthology.orchestra.run.smithi076.stdout:1994 writesame 0x7a66908 thru 0x7a6a5fb data_size 0x4(0x3cf4 bytes) 2024-09-14T22:53:50.974 INFO:teuthology.orchestra.run.smithi076.stdout:1995 trunc from 0xd96d5cf to 0xe7a5884 2024-09-14T22:53:50.980 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:50.980 INFO:teuthology.orchestra.run.smithi076.stdout:1999 read 0xba4cad9 thru 0xba5bf19 (0xf441 bytes) 2024-09-14T22:53:50.980 INFO:teuthology.orchestra.run.smithi076.stdout:2000 write 0xa455ecd thru 0xa464c2d (0xed61 bytes) 2024-09-14T22:53:50.984 INFO:teuthology.orchestra.run.smithi076.stdout:2001 write 0xe5a08a1 thru 0xe5a3eb4 (0x3614 bytes) 2024-09-14T22:53:50.986 INFO:teuthology.orchestra.run.smithi076.stdout:2002 read 0x7d1a5df thru 0x7d23223 (0x8c45 bytes) 2024-09-14T22:53:50.987 INFO:teuthology.orchestra.run.smithi076.stdout:2004 writesame 0xd4d701c thru 0xd4e0c7b data_size 0x8b(0x9c60 bytes) 2024-09-14T22:53:50.995 INFO:teuthology.orchestra.run.smithi076.stdout:2005 trunc from 0xe7a5884 to 0xa45e2a3 2024-09-14T22:53:51.007 INFO:teuthology.orchestra.run.smithi076.stdout:2006 clone 19 order 23 su 1048576 sc 5 2024-09-14T22:53:51.007 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:51.006+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:51.055 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:51.055 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:51.055 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:51.056 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:51.056 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:51.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:51.108 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:51.106+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:51.208 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:51.207+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:51.211 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:51.211 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.359040749999998, 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.358103159999999, 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:53:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:53:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.358011797, 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:53:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:53:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.357366912, 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.356879533000001, 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:53:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:53:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.355992258000001, 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:53:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.355399834, 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:53:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:53:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.354667018000001, 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.354567869, 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.354249411000001, 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.353739916999999, 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:53:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:53:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.353682058, 2024-09-14T22:53:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:53:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:53:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.352889435000002, 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.352213996, 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:53:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.352080686000001, 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:53:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:53:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:53:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.351463763000002, 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:53:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.351396906000002, 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:53:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:53:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.350946095000001, 2024-09-14T22:53:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:53:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:53:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.350825862000001, 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:53:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.349507735, 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:53:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:51.306 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:51.306 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:51.307 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:51.309 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:51.308+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:51.410 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:51.408+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:51.454 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:51.454 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:51.454 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:51.455 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:51.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:51.469 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:51.470 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:51.488 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:51.488 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:51.505 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:51.505 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:51.511 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:51.510+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:51.521 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:51.521 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:51.537 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:51.538 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:51.554 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:51.555 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:51.569 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:51.569 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:51.571 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:51.571 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:51.578 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:51.588 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:51.589 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:51.612 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:51.611+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:51.691 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:51.691 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:51.700 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:51.743 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:51.742+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:51.802 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:51.802 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:51.810 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:51.852 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:51.850+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:51.914 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:51.913+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:51.942 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:53:51.959 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:51.959 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:51.969 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:52.053 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:52.052+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:52.113 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T22:53:52.116 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:52.115+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:52.120 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:52.121 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:52.133 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:52.217 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:52.216+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:52.293 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:52.293 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:52.302 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:52.318 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:52.317+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:52.417 INFO:teuthology.orchestra.run.smithi076.stdout:2007 read 0x6cc615b thru 0x6cd209d (0xbf43 bytes) 2024-09-14T22:53:52.418 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:52.417+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:52.424 INFO:teuthology.orchestra.run.smithi076.stdout:2010 trunc from 0xa45e2a3 to 0x5e8837c 2024-09-14T22:53:52.448 INFO:teuthology.orchestra.run.smithi076.stdout:2012 punch from 0x5adbdb6 to 0x5ade0fd, (0x2347 bytes) 2024-09-14T22:53:52.461 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:52.461 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:52.465 INFO:teuthology.orchestra.run.smithi076.stdout:2013 writesame 0xe5a3afb thru 0xe5a4d2d data_size 0x1(0x1233 bytes) 2024-09-14T22:53:52.465 INFO:teuthology.orchestra.run.smithi076.stdout:2015 write 0x953689e thru 0x95383ae (0x1b11 bytes) 2024-09-14T22:53:52.468 INFO:teuthology.orchestra.run.smithi076.stdout:2016 write 0xee4735 thru 0xef003d (0xb909 bytes) 2024-09-14T22:53:52.470 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:52.472 INFO:teuthology.orchestra.run.smithi076.stdout:2017 punch from 0xe07b78a to 0xe07f052, (0x38c8 bytes) 2024-09-14T22:53:52.472 INFO:teuthology.orchestra.run.smithi076.stdout:2018 write 0x70597b2 thru 0x7064e37 (0xb686 bytes) 2024-09-14T22:53:52.474 INFO:teuthology.orchestra.run.smithi076.stdout:2021 read 0x3b84e6 thru 0x3bee9e (0x69b9 bytes) 2024-09-14T22:53:52.474 INFO:teuthology.orchestra.run.smithi076.stdout:2023 clone 20 order 22 su 65536 sc 8 2024-09-14T22:53:52.519 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:52.518+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:52.575 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:52.576 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:52.584 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:52.628 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:52.627+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:52.693 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:52.693 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:52.701 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:52.742 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:52.741+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:52.821 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:52.820+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:52.845 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:52.845 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:52.854 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:52.922 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:52.921+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:53.011 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:53.016 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone19 from 0xe5a4d2e (overlap 0x5e8837c) to 0x1c5c10b 2024-09-14T22:53:53.023 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:53.022+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:53.090 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T22:53:53.119 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:53.119 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:53.125 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:53.123+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:53.128 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:53.226 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:53.224+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:53.230 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:53.230 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:53.239 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:53.303 INFO:teuthology.orchestra.run.smithi076.stdout:2024 write 0x8a716af thru 0x8a7478e (0x30e0 bytes) 2024-09-14T22:53:53.314 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:53.314 INFO:teuthology.orchestra.run.smithi076.stdout:2026 write 0xc54ce7d thru 0xc553579 (0x66fd bytes) 2024-09-14T22:53:53.327 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:53.325+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:53.334 INFO:teuthology.orchestra.run.smithi076.stdout:2028 trunc from 0xe5a4d2e to 0x25d5a06 2024-09-14T22:53:53.349 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:53.349 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:53.358 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:53.360 INFO:teuthology.orchestra.run.smithi076.stdout:2029 punch from 0xda8d6b to 0xdaf5c4, (0x6859 bytes) 2024-09-14T22:53:53.360 INFO:teuthology.orchestra.run.smithi076.stdout:2030 read 0xf0bc5d thru 0xf0c9d2 (0xd76 bytes) 2024-09-14T22:53:53.360 INFO:teuthology.orchestra.run.smithi076.stdout:2031 read 0x2264016 thru 0x22663ff (0x23ea bytes) 2024-09-14T22:53:53.360 INFO:teuthology.orchestra.run.smithi076.stdout:2032 read 0x13f9023 thru 0x1408750 (0xf72e bytes) 2024-09-14T22:53:53.363 INFO:teuthology.orchestra.run.smithi076.stdout:2033 trunc from 0x25d5a06 to 0x3c1dd50 2024-09-14T22:53:53.368 INFO:teuthology.orchestra.run.smithi076.stdout:2034 read 0x10a0faa thru 0x10b0006 (0xf05d bytes) 2024-09-14T22:53:53.368 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:53.368 INFO:teuthology.orchestra.run.smithi076.stdout:2036 read 0x14822fc thru 0x1486153 (0x3e58 bytes) 2024-09-14T22:53:53.371 INFO:teuthology.orchestra.run.smithi076.stdout:2037 trunc from 0x3c1dd50 to 0x1b5571a 2024-09-14T22:53:53.416 INFO:teuthology.orchestra.run.smithi076.stdout:2039 punch from 0x892659 to 0x89887c, (0x6223 bytes) 2024-09-14T22:53:53.429 INFO:teuthology.orchestra.run.smithi076.stdout:2040 writesame 0x890df7e thru 0x8912ec2 data_size 0xd(0x4f45 bytes) 2024-09-14T22:53:53.429 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:53.435 INFO:teuthology.orchestra.run.smithi076.stdout:2043 trunc from 0x8912ec3 to 0xdc54019 2024-09-14T22:53:53.441 INFO:teuthology.orchestra.run.smithi076.stdout:2045 write 0x6221b8c thru 0x62264e1 (0x4956 bytes) 2024-09-14T22:53:53.442 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:53.441+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:53.444 INFO:teuthology.orchestra.run.smithi076.stdout:2046 punch from 0x110e4e6 to 0x111d3f8, (0xef12 bytes) 2024-09-14T22:53:53.444 INFO:teuthology.orchestra.run.smithi076.stdout:2047 write 0x4a25f32 thru 0x4a33249 (0xd318 bytes) 2024-09-14T22:53:53.448 INFO:teuthology.orchestra.run.smithi076.stdout:2048 read 0x8b4ea68 thru 0x8b5c245 (0xd7de bytes) 2024-09-14T22:53:53.448 INFO:teuthology.orchestra.run.smithi076.stdout:2050 read 0x1fd6788 thru 0x1fe1c70 (0xb4e9 bytes) 2024-09-14T22:53:53.448 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:53.448 INFO:teuthology.orchestra.run.smithi076.stdout:2052 punch from 0xc3dcb67 to 0xc3e527e, (0x8717 bytes) 2024-09-14T22:53:53.457 INFO:teuthology.orchestra.run.smithi076.stdout:2053 trunc from 0xdc54019 to 0x6291f52 2024-09-14T22:53:53.464 INFO:teuthology.orchestra.run.smithi076.stdout:2054 write 0x3809389 thru 0x380f121 (0x5d99 bytes) 2024-09-14T22:53:53.467 INFO:teuthology.orchestra.run.smithi076.stdout:2055 write 0x38c8da6 thru 0x38cbfc2 (0x321d bytes) 2024-09-14T22:53:53.474 INFO:teuthology.orchestra.run.smithi076.stdout:2056 trunc from 0x6291f52 to 0x1de3136 2024-09-14T22:53:53.489 INFO:teuthology.orchestra.run.smithi076.stdout:2057 read 0x43fbb4 thru 0x4403bc (0x809 bytes) 2024-09-14T22:53:53.489 INFO:teuthology.orchestra.run.smithi076.stdout:2058 write 0x1966214 thru 0x196890a (0x26f7 bytes) 2024-09-14T22:53:53.491 INFO:teuthology.orchestra.run.smithi076.stdout:2060 writesame 0x17bb479 thru 0x17c5406 data_size 0xd(0x9f8e bytes) 2024-09-14T22:53:53.494 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:53.494 INFO:teuthology.orchestra.run.smithi076.stdout:2063 punch from 0xcc5482 to 0xcc8889, (0x3407 bytes) 2024-09-14T22:53:53.503 INFO:teuthology.orchestra.run.smithi076.stdout:2064 write 0x4be9e7e thru 0x4bf45e3 (0xa766 bytes) 2024-09-14T22:53:53.507 INFO:teuthology.orchestra.run.smithi076.stdout:2065 write 0x4705553 thru 0x470556a (0x18 bytes) 2024-09-14T22:53:53.512 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:53.512 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:53.521 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:53.526 INFO:teuthology.orchestra.run.smithi076.stdout:2066 write 0xd4dd989 thru 0xd4e6a29 (0x90a1 bytes) 2024-09-14T22:53:53.532 INFO:teuthology.orchestra.run.smithi076.stdout:2067 trunc from 0xd4e6a2a to 0xc9aaa46 2024-09-14T22:53:53.538 INFO:teuthology.orchestra.run.smithi076.stdout:2068 punch from 0x2df5460 to 0x2dfbb4e, (0x66ee bytes) 2024-09-14T22:53:53.538 INFO:teuthology.orchestra.run.smithi076.stdout:2069 read 0x125ecd9 thru 0x12653e5 (0x670d bytes) 2024-09-14T22:53:53.539 INFO:teuthology.orchestra.run.smithi076.stdout:2070 write 0x7e25c05 thru 0x7e25e44 (0x240 bytes) 2024-09-14T22:53:53.542 INFO:teuthology.orchestra.run.smithi076.stdout:2071 punch from 0x1226830 to 0x122dd2f, (0x74ff bytes) 2024-09-14T22:53:53.542 INFO:teuthology.orchestra.run.smithi076.stdout:2073 writesame 0xb7ae878 thru 0xb7aed34 data_size 0x1(0x4bd bytes) 2024-09-14T22:53:53.545 INFO:teuthology.orchestra.run.smithi076.stdout:2075 write 0xc231ec thru 0xc2467c (0x1491 bytes) 2024-09-14T22:53:53.546 INFO:teuthology.orchestra.run.smithi076.stdout:2076 read 0x2606131 thru 0x2612f5a (0xce2a bytes) 2024-09-14T22:53:53.558 INFO:teuthology.orchestra.run.smithi076.stdout:2077 trunc from 0xc9aaa46 to 0x2b5e7fa 2024-09-14T22:53:53.562 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:53.561+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:53.581 INFO:teuthology.orchestra.run.smithi076.stdout:2078 writesame 0x8214912 thru 0x821b7d4 data_size 0x35(0x6ec3 bytes) 2024-09-14T22:53:53.581 INFO:teuthology.orchestra.run.smithi076.stdout:2079 punch from 0x618bc82 to 0x619086c, (0x4bea bytes) 2024-09-14T22:53:53.581 INFO:teuthology.orchestra.run.smithi076.stdout:2081 read 0x45619ee thru 0x456a52d (0x8b40 bytes) 2024-09-14T22:53:53.581 INFO:teuthology.orchestra.run.smithi076.stdout:2082 read 0x7dd094c thru 0x7dd0b51 (0x206 bytes) 2024-09-14T22:53:53.581 INFO:teuthology.orchestra.run.smithi076.stdout:2084 punch from 0x2979dec to 0x2984ec0, (0xb0d4 bytes) 2024-09-14T22:53:53.581 INFO:teuthology.orchestra.run.smithi076.stdout:2086 writesame 0x4152c78 thru 0x4153170 data_size 0x1(0x4f9 bytes) 2024-09-14T22:53:53.591 INFO:teuthology.orchestra.run.smithi076.stdout:2088 trunc from 0x821b7d5 to 0x28b8668 2024-09-14T22:53:53.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:53.610 INFO:teuthology.orchestra.run.smithi076.stdout:2089 writesame 0x5e6fe8f thru 0x5e79154 data_size 0x1(0x92c6 bytes) 2024-09-14T22:53:53.610 INFO:teuthology.orchestra.run.smithi076.stdout:2091 punch from 0x1653735 to 0x16608d4, (0xd19f bytes) 2024-09-14T22:53:53.616 INFO:teuthology.orchestra.run.smithi076.stdout:2092 trunc from 0x5e79155 to 0x113d000 2024-09-14T22:53:53.630 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:53.629+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:53.640 INFO:teuthology.orchestra.run.smithi076.stdout:2093 write 0x9207d4b thru 0x920baea (0x3da0 bytes) 2024-09-14T22:53:53.643 INFO:teuthology.orchestra.run.smithi076.stdout:2094 read 0x2f86728 thru 0x2f8b6fd (0x4fd6 bytes) 2024-09-14T22:53:53.644 INFO:teuthology.orchestra.run.smithi076.stdout:2095 read 0x57ad60b thru 0x57b3580 (0x5f76 bytes) 2024-09-14T22:53:53.644 INFO:teuthology.orchestra.run.smithi076.stdout:2096 writesame 0x30b77f0 thru 0x30c3c90 data_size 0x33(0xc4a1 bytes) 2024-09-14T22:53:53.652 INFO:teuthology.orchestra.run.smithi076.stdout:2097 trunc from 0x920baeb to 0x5568eaf 2024-09-14T22:53:53.663 INFO:teuthology.orchestra.run.smithi076.stdout:2098 punch from 0x1bc4f87 to 0x1bd26b8, (0xd731 bytes) 2024-09-14T22:53:53.663 INFO:teuthology.orchestra.run.smithi076.stdout:2099 write 0x29fa63f thru 0x29fc2d6 (0x1c98 bytes) 2024-09-14T22:53:53.665 INFO:teuthology.orchestra.run.smithi076.stdout:2101 writesame 0x2d4491 thru 0x2e3fe5 data_size 0x9(0xfb55 bytes) 2024-09-14T22:53:53.670 INFO:teuthology.orchestra.run.smithi076.stdout:2102 read 0x4755080 thru 0x4759ca8 (0x4c29 bytes) 2024-09-14T22:53:53.670 INFO:teuthology.orchestra.run.smithi076.stdout:2103 read 0x7a49da thru 0x7aa099 (0x56c0 bytes) 2024-09-14T22:53:53.671 INFO:teuthology.orchestra.run.smithi076.stdout:2105 write 0x9923ec thru 0x99ff1a (0xdb2f bytes) 2024-09-14T22:53:53.672 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:53.672 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:53.673 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:53.675 INFO:teuthology.orchestra.run.smithi076.stdout:2107 trunc from 0x5568eaf to 0x77b6df2 2024-09-14T22:53:53.681 INFO:teuthology.orchestra.run.smithi076.stdout:2108 read 0x4ebac8f thru 0x4ec6e1c (0xc18e bytes) 2024-09-14T22:53:53.682 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:53.686 INFO:teuthology.orchestra.run.smithi076.stdout:2109 trunc from 0x77b6df2 to 0xc138fff 2024-09-14T22:53:53.692 INFO:teuthology.orchestra.run.smithi076.stdout:2110 write 0x4ecafba thru 0x4ed5a42 (0xaa89 bytes) 2024-09-14T22:53:53.696 INFO:teuthology.orchestra.run.smithi076.stdout:2111 punch from 0x67db554 to 0x67dfdbe, (0x486a bytes) 2024-09-14T22:53:53.696 INFO:teuthology.orchestra.run.smithi076.stdout:2112 clone 21 order 25 su 2097152 sc 7 2024-09-14T22:53:53.731 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:53.730+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:53.732 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:53.732 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:53.749 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:53.749 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:53.764 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:53.764 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:53.780 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:53.780 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:53.786 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:53.786 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:53.795 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:53.797 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:53.798 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:53.813 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:53.813 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:53.830 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:53.831 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:53.832 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:53.830+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:53.847 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:53.847 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:53.899 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:53.899 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:53.899 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:53.907 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:53.933 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:53.932+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:53.958 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:53:54.017 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:53:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:53:54.018 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:54.021 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone20 from 0xc138fff (overlap 0x113d000) to 0x6e5333 2024-09-14T22:53:54.028 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:54.034 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:54.032+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:54.099 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T22:53:54.135 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:54.134+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:54.137 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:54.137 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:53:54.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:53:54.137 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:53:54.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.627458159, 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:53:54.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:53:54.144 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762800303999999, 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:53:54.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:53:54.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:53:54.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762694367999998, 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.149 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:53:54.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:53:54.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762619666999999, 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:53:54.157 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762534186, 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:53:54.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:53:54.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:53:54.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762387799999999, 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.162 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:53:54.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:53:54.166 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762337800000001, 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:53:54.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762174636000001, 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.171 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:53:54.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:53:54.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.175 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762136326, 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:53:54.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.762002657, 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.184 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.761922089999999, 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:53:54.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.761825153, 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.192 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.761773598000001, 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:53:54.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:53:54.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:53:54.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.758648968999999, 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:53:54.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:53:54.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.758517355999999, 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:53:54.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:53:54.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.758230944000001, 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118419 3.15 3.2a32cf5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377105+0000", 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.758129179000001, 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050052747000000002, 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.210 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118419 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377105+0000", 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377105+0000", 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377105+0000", 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8100000000000001e-07 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377110+0000", 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3200000000000001e-06 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377121+0000", 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1131999999999999e-05 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377128+0000", 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2760000000000003e-06 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427153+0000", 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050024787000000001 2024-09-14T22:53:54.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427157+0000", 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3510000000000004e-06 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118420 3.15 3.2a32cf5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377140+0000", 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.758094100000001, 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050044040999999997, 2024-09-14T22:53:54.214 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118420 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377140+0000", 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377140+0000", 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377140+0000", 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9200000000000001e-07 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377143+0000", 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7159999999999999e-06 2024-09-14T22:53:54.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377150+0000", 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6950000000000003e-06 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377155+0000", 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2360000000000004e-06 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427176+0000", 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050020792000000001 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427184+0000", 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1100000000000003e-06 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118421 3.15 3.2a32cf5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377162+0000", 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.758071986000001, 2024-09-14T22:53:54.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050048141999999997, 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118421 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377162+0000", 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377162+0000", 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377162+0000", 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377165+0000", 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2670000000000001e-06 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377171+0000", 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1750000000000002e-06 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377187+0000", 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6337e-05 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427203+0000", 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050016400000000003 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427210+0000", 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5139999999999998e-06 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.222 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118432 3.15 3.2a32cf5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377584+0000", 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.757650139999999, 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050039961000000001, 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118432 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377584+0000", 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377584+0000", 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377689+0000", 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010526699999999999 2024-09-14T22:53:54.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377693+0000", 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.681e-06 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377701+0000", 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9359999999999999e-06 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377707+0000", 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.998e-06 2024-09-14T22:53:54.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427618+0000", 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049911075999999999 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427624+0000", 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.003e-06 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:53:54.226 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:54.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:54.229 INFO:teuthology.orchestra.run.smithi076.stdout:2115 read 0x2d2a400 thru 0x2d305bb (0x61bc bytes) 2024-09-14T22:53:54.229 INFO:teuthology.orchestra.run.smithi076.stdout:2116 writesame 0x18d9879 thru 0x18d9993 data_size 0x1(0x11b bytes) 2024-09-14T22:53:54.229 INFO:teuthology.orchestra.run.smithi076.stdout:2117 read 0x8d1764b thru 0x8d1f98a (0x8340 bytes) 2024-09-14T22:53:54.229 INFO:teuthology.orchestra.run.smithi076.stdout:2118 read 0x508668f thru 0x5087ed0 (0x1842 bytes) 2024-09-14T22:53:54.229 INFO:teuthology.orchestra.run.smithi076.stdout:2119 read 0x8e6d456 thru 0x8e70ad9 (0x3684 bytes) 2024-09-14T22:53:54.230 INFO:teuthology.orchestra.run.smithi076.stdout:2120 write 0xcd1a918 thru 0xcd1f805 (0x4eee bytes) 2024-09-14T22:53:54.230 INFO:teuthology.orchestra.run.smithi076.stdout:2121 read 0x2f1c092 thru 0x2f1c81a (0x789 bytes) 2024-09-14T22:53:54.230 INFO:teuthology.orchestra.run.smithi076.stdout:2122 write 0x981aa4e thru 0x98289d3 (0xdf86 bytes) 2024-09-14T22:53:54.230 INFO:teuthology.orchestra.run.smithi076.stdout:2123 trunc from 0xcd1f806 to 0x4149b29 2024-09-14T22:53:54.230 INFO:teuthology.orchestra.run.smithi076.stdout:2124 trunc from 0x4149b29 to 0xcf213e8 2024-09-14T22:53:54.230 INFO:teuthology.orchestra.run.smithi076.stdout:2125 trunc from 0xcf213e8 to 0x62af1d6 2024-09-14T22:53:54.232 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.232 INFO:teuthology.orchestra.run.smithi076.stdout:2128 read 0x24e2667 thru 0x24edc24 (0xb5be bytes) 2024-09-14T22:53:54.232 INFO:teuthology.orchestra.run.smithi076.stdout:2129 read 0x2c88548 thru 0x2c8e2a9 (0x5d62 bytes) 2024-09-14T22:53:54.232 INFO:teuthology.orchestra.run.smithi076.stdout:2130 punch from 0x4f79644 to 0x4f8741b, (0xddd7 bytes) 2024-09-14T22:53:54.236 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:54.234+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:54.237 INFO:teuthology.orchestra.run.smithi076.stdout:2131 trunc from 0x62af1d6 to 0xa771110 2024-09-14T22:53:54.242 INFO:teuthology.orchestra.run.smithi076.stdout:2132 read 0x9551e75 thru 0x9556b36 (0x4cc2 bytes) 2024-09-14T22:53:54.242 INFO:teuthology.orchestra.run.smithi076.stdout:2133 punch from 0x3b3d62c to 0x3b41c36, (0x460a bytes) 2024-09-14T22:53:54.243 INFO:teuthology.orchestra.run.smithi076.stdout:2134 writesame 0x4ff630f thru 0x5005dfc data_size 0x1(0xfaee bytes) 2024-09-14T22:53:54.274 INFO:teuthology.orchestra.run.smithi076.stdout:2136 read 0x565d349 thru 0x566a267 (0xcf1f bytes) 2024-09-14T22:53:54.275 INFO:teuthology.orchestra.run.smithi076.stdout:2137 read 0x3a68319 thru 0x3a70552 (0x823a bytes) 2024-09-14T22:53:54.275 INFO:teuthology.orchestra.run.smithi076.stdout:2138 read 0x340eca thru 0x34ef74 (0xe0ab bytes) 2024-09-14T22:53:54.276 INFO:teuthology.orchestra.run.smithi076.stdout:2139 punch from 0x7f6f47b to 0x7f776ed, (0x8272 bytes) 2024-09-14T22:53:54.276 INFO:teuthology.orchestra.run.smithi076.stdout:2140 punch from 0x726bd3d to 0x7270cd9, (0x4f9c bytes) 2024-09-14T22:53:54.276 INFO:teuthology.orchestra.run.smithi076.stdout:2143 write 0x4f8bdc3 thru 0x4f8d2c5 (0x1503 bytes) 2024-09-14T22:53:54.278 INFO:teuthology.orchestra.run.smithi076.stdout:2144 punch from 0x1f6e60c to 0x1f76750, (0x8144 bytes) 2024-09-14T22:53:54.278 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.279 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.279 INFO:teuthology.orchestra.run.smithi076.stdout:2148 write 0x8d2a689 thru 0x8d2c7da (0x2152 bytes) 2024-09-14T22:53:54.280 INFO:teuthology.orchestra.run.smithi076.stdout:2149 punch from 0x64c2cd to 0x65c06a, (0xfd9d bytes) 2024-09-14T22:53:54.288 INFO:teuthology.orchestra.run.smithi076.stdout:2150 trunc from 0xa771110 to 0x42254eb 2024-09-14T22:53:54.305 INFO:teuthology.orchestra.run.smithi076.stdout:2151 trunc from 0x42254eb to 0xafe9716 2024-09-14T22:53:54.323 INFO:teuthology.orchestra.run.smithi076.stdout:2152 trunc from 0xafe9716 to 0x116c4d 2024-09-14T22:53:54.337 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:54.335+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:54.349 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:54.349 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:54.349 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:54.349 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:54.350 INFO:teuthology.orchestra.run.smithi076.stdout:2153 write 0x4ca4d93 thru 0x4ca54c5 (0x733 bytes) 2024-09-14T22:53:54.353 INFO:teuthology.orchestra.run.smithi076.stdout:2154 writesame 0x43c56ab thru 0x43d4dba data_size 0x86(0xf710 bytes) 2024-09-14T22:53:54.359 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:54.364 INFO:teuthology.orchestra.run.smithi076.stdout:2155 writesame 0x624a31e thru 0x6254c25 data_size 0x24(0xa908 bytes) 2024-09-14T22:53:54.366 INFO:teuthology.orchestra.run.smithi076.stdout:2156 read 0x54cd2e2 thru 0x54d8344 (0xb063 bytes) 2024-09-14T22:53:54.370 INFO:teuthology.orchestra.run.smithi076.stdout:2157 trunc from 0x6254c26 to 0xa314258 2024-09-14T22:53:54.383 INFO:teuthology.orchestra.run.smithi076.stdout:2159 trunc from 0xa314258 to 0x3d13ddc 2024-09-14T22:53:54.389 INFO:teuthology.orchestra.run.smithi076.stdout:2160 read 0x2c760e3 thru 0x2c79405 (0x3323 bytes) 2024-09-14T22:53:54.389 INFO:teuthology.orchestra.run.smithi076.stdout:2162 read 0x2ef36f4 thru 0x2f010db (0xd9e8 bytes) 2024-09-14T22:53:54.393 INFO:teuthology.orchestra.run.smithi076.stdout:2163 trunc from 0x3d13ddc to 0xbdf537 2024-09-14T22:53:54.417 INFO:teuthology.orchestra.run.smithi076.stdout:2164 write 0xac3462e thru 0xac4006a (0xba3d bytes) 2024-09-14T22:53:54.420 INFO:teuthology.orchestra.run.smithi076.stdout:2165 read 0x58dfc07 thru 0x58ecbd2 (0xcfcc bytes) 2024-09-14T22:53:54.420 INFO:teuthology.orchestra.run.smithi076.stdout:2166 write 0x96777ff thru 0x96849cd (0xd1cf bytes) 2024-09-14T22:53:54.422 INFO:teuthology.orchestra.run.smithi076.stdout:2167 read 0x32ce9f0 thru 0x32dbb5e (0xd16f bytes) 2024-09-14T22:53:54.427 INFO:teuthology.orchestra.run.smithi076.stdout:2168 write 0xb305606 thru 0xb310a1e (0xb419 bytes) 2024-09-14T22:53:54.428 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.428 INFO:teuthology.orchestra.run.smithi076.stdout:2170 read 0x47d1765 thru 0x47da00c (0x88a8 bytes) 2024-09-14T22:53:54.429 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.429 INFO:teuthology.orchestra.run.smithi076.stdout:2172 read 0x88958b3 thru 0x8896ace (0x121c bytes) 2024-09-14T22:53:54.429 INFO:teuthology.orchestra.run.smithi076.stdout:2173 writesame 0xa769b4e thru 0xa772758 data_size 0x1(0x8c0b bytes) 2024-09-14T22:53:54.437 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:54.435+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:54.449 INFO:teuthology.orchestra.run.smithi076.stdout:2177 read 0x5d0b122 thru 0x5d147ca (0x96a9 bytes) 2024-09-14T22:53:54.449 INFO:teuthology.orchestra.run.smithi076.stdout:2178 write 0x332cb5b thru 0x33394aa (0xc950 bytes) 2024-09-14T22:53:54.453 INFO:teuthology.orchestra.run.smithi076.stdout:2181 write 0x2b50498 thru 0x2b5206b (0x1bd4 bytes) 2024-09-14T22:53:54.454 INFO:teuthology.orchestra.run.smithi076.stdout:2182 write 0x466aa9f thru 0x466c329 (0x188b bytes) 2024-09-14T22:53:54.458 INFO:teuthology.orchestra.run.smithi076.stdout:2183 trunc from 0xb310a1f to 0x90c2d2f 2024-09-14T22:53:54.466 INFO:teuthology.orchestra.run.smithi076.stdout:2184 writesame 0x6d9585d thru 0x6da1b4d data_size 0xf(0xc2f1 bytes) 2024-09-14T22:53:54.481 INFO:teuthology.orchestra.run.smithi076.stdout:2185 write 0xe5dd41b thru 0xe5e02f1 (0x2ed7 bytes) 2024-09-14T22:53:54.481 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:54.481 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:54.481 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:54.481 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:54.482 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:54.485 INFO:teuthology.orchestra.run.smithi076.stdout:2186 read 0x8d8349f thru 0x8d84a47 (0x15a9 bytes) 2024-09-14T22:53:54.490 INFO:teuthology.orchestra.run.smithi076.stdout:2187 trunc from 0xe5e02f2 to 0xa668fdc 2024-09-14T22:53:54.492 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:54.505 INFO:teuthology.orchestra.run.smithi076.stdout:2189 read 0x80f6289 thru 0x80faac4 (0x483c bytes) 2024-09-14T22:53:54.514 INFO:teuthology.orchestra.run.smithi076.stdout:2190 write 0xe87d091 thru 0xe886872 (0x97e2 bytes) 2024-09-14T22:53:54.519 INFO:teuthology.orchestra.run.smithi076.stdout:2191 punch from 0x429dc45 to 0x42a59e5, (0x7da0 bytes) 2024-09-14T22:53:54.521 INFO:teuthology.orchestra.run.smithi076.stdout:2192 trunc from 0xe886873 to 0xc4e23a3 2024-09-14T22:53:54.536 INFO:teuthology.orchestra.run.smithi076.stdout:2194 write 0xe0fc07 thru 0xe10a00 (0xdfa bytes) 2024-09-14T22:53:54.537 INFO:teuthology.orchestra.run.smithi076.stdout:2195 writesame 0x7178740 thru 0x71790e7 data_size 0xc(0x9a8 bytes) 2024-09-14T22:53:54.538 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:54.536+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:54.539 INFO:teuthology.orchestra.run.smithi076.stdout:2196 write 0x94a5c7 thru 0x958ee8 (0xe922 bytes) 2024-09-14T22:53:54.540 INFO:teuthology.orchestra.run.smithi076.stdout:2198 write 0x80b045b thru 0x80b91c6 (0x8d6c bytes) 2024-09-14T22:53:54.558 INFO:teuthology.orchestra.run.smithi076.stdout:2199 trunc from 0xc4e23a3 to 0xb06cc5 2024-09-14T22:53:54.574 INFO:teuthology.orchestra.run.smithi076.stdout:2200 trunc from 0xb06cc5 to 0x69281aa 2024-09-14T22:53:54.579 INFO:teuthology.orchestra.run.smithi076.stdout:2202 read 0x1b878c7 thru 0x1b87c14 (0x34e bytes) 2024-09-14T22:53:54.579 INFO:teuthology.orchestra.run.smithi076.stdout:2206 read 0x474109 thru 0x4826c3 (0xe5bb bytes) 2024-09-14T22:53:54.580 INFO:teuthology.orchestra.run.smithi076.stdout:2207 punch from 0x9baa71 to 0x9c1b5a, (0x70e9 bytes) 2024-09-14T22:53:54.580 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.594 INFO:teuthology.orchestra.run.smithi076.stdout:2209 writesame 0xdb9ae0c thru 0xdba60a3 data_size 0x7f(0xb298 bytes) 2024-09-14T22:53:54.595 INFO:teuthology.orchestra.run.smithi076.stdout:2210 read 0x4ca34ef thru 0x4ca4ac0 (0x15d2 bytes) 2024-09-14T22:53:54.608 INFO:teuthology.orchestra.run.smithi076.stdout:2211 trunc from 0xdba60a4 to 0x32ca926 2024-09-14T22:53:54.615 INFO:teuthology.orchestra.run.smithi076.stdout:2213 read 0x15310c4 thru 0x153e92a (0xd867 bytes) 2024-09-14T22:53:54.623 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:54.623 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.771485407, 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.624 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:53:54.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.628 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.770547817000001, 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.770456454000001, 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.633 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:53:54.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:53:54.637 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.769811569000002, 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.638 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.639 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:53:54.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.769324189999999, 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.643 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.644 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:53:54.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.768436914999999, 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.648 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.767844490999998, 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.653 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:53:54.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.767111674999999, 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.658 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:53:54.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:53:54.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:53:54.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:53:54.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:53:54.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.662 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.767012525999998, 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:54.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.667 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.766694068, 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.766184574, 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.672 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:53:54.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.676 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.766126714999999, 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.677 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:53:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.682 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.682 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:54.682 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.765334092, 2024-09-14T22:53:54.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:53:54.682 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.682 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.682 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:53:54.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:53:54.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.686 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.764658653000001, 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.687 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.764525342999999, 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.691 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:53:54.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:53:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.76390842, 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:53:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:53:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:53:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.763841563, 2024-09-14T22:53:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.763390751999999, 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:53:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.763270518999999, 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:53:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:53:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.761952392000001, 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:53:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:54.718 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:54.719 INFO:teuthology.orchestra.run.smithi076.stdout:2215 write 0x842b491 thru 0x8433743 (0x82b3 bytes) 2024-09-14T22:53:54.719 INFO:teuthology.orchestra.run.smithi076.stdout:2216 read 0x42f9d27 thru 0x430782c (0xdb06 bytes) 2024-09-14T22:53:54.719 INFO:teuthology.orchestra.run.smithi076.stdout:2217 writesame 0x104f009 thru 0x105e273 data_size 0x1(0xf26b bytes) 2024-09-14T22:53:54.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:54.722 INFO:teuthology.orchestra.run.smithi076.stdout:2218 trunc from 0x8433744 to 0xed9956 2024-09-14T22:53:54.722 INFO:teuthology.orchestra.run.smithi076.stdout:2219 writesame 0x2b067b3 thru 0x2b0ca62 data_size 0x4(0x62b0 bytes) 2024-09-14T22:53:54.722 INFO:teuthology.orchestra.run.smithi076.stdout:2221 writesame 0x67cf5f6 thru 0x67d090e data_size 0x1(0x1319 bytes) 2024-09-14T22:53:54.722 INFO:teuthology.orchestra.run.smithi076.stdout:2222 trunc from 0x67d090f to 0xe8e6ee6 2024-09-14T22:53:54.722 INFO:teuthology.orchestra.run.smithi076.stdout:2223 read 0x50bba7 thru 0x50c89f (0xcf9 bytes) 2024-09-14T22:53:54.722 INFO:teuthology.orchestra.run.smithi076.stdout:2224 punch from 0xa38fb9d to 0xa39ccf4, (0xd157 bytes) 2024-09-14T22:53:54.722 INFO:teuthology.orchestra.run.smithi076.stdout:2225 read 0x8c67e48 thru 0x8c6c4c0 (0x4679 bytes) 2024-09-14T22:53:54.722 INFO:teuthology.orchestra.run.smithi076.stdout:2226 write 0x1cff346 thru 0x1d01fcc (0x2c87 bytes) 2024-09-14T22:53:54.723 INFO:teuthology.orchestra.run.smithi076.stdout:2227 write 0x3e95169 thru 0x3ea387c (0xe714 bytes) 2024-09-14T22:53:54.723 INFO:teuthology.orchestra.run.smithi076.stdout:2228 read 0xc4099a5 thru 0xc40d25d (0x38b9 bytes) 2024-09-14T22:53:54.723 INFO:teuthology.orchestra.run.smithi076.stdout:2230 read 0xd2cc734 thru 0xd2da046 (0xd913 bytes) 2024-09-14T22:53:54.723 INFO:teuthology.orchestra.run.smithi076.stdout:2231 writesame 0xafc14e6 thru 0xafc4e08 data_size 0x1(0x3923 bytes) 2024-09-14T22:53:54.723 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:54.717+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:54.739 INFO:teuthology.orchestra.run.smithi076.stdout:2232 trunc from 0xe8e6ee6 to 0x7caaad0 2024-09-14T22:53:54.758 INFO:teuthology.orchestra.run.smithi076.stdout:2233 write 0xc813f14 thru 0xc81acb6 (0x6da3 bytes) 2024-09-14T22:53:54.759 INFO:teuthology.orchestra.run.smithi076.stdout:2235 write 0x210ccb9 thru 0x210d446 (0x78e bytes) 2024-09-14T22:53:54.760 INFO:teuthology.orchestra.run.smithi076.stdout:2239 writesame 0x61e2e0f thru 0x61ef4d9 data_size 0x1(0xc6cb bytes) 2024-09-14T22:53:54.788 INFO:teuthology.orchestra.run.smithi076.stdout:2240 writesame 0xb250db9 thru 0xb25f759 data_size 0x1(0xe9a1 bytes) 2024-09-14T22:53:54.821 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:54.818+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:54.824 INFO:teuthology.orchestra.run.smithi076.stdout:2241 punch from 0x3d4f018 to 0x3d5debd, (0xeea5 bytes) 2024-09-14T22:53:54.832 INFO:teuthology.orchestra.run.smithi076.stdout:2242 write 0xe33d726 thru 0xe33f10a (0x19e5 bytes) 2024-09-14T22:53:54.833 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:54.833 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:54.834 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:54.834 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:54.834 INFO:teuthology.orchestra.run.smithi076.stdout:2243 read 0x1abded8 thru 0x1ac61bd (0x82e6 bytes) 2024-09-14T22:53:54.835 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.835 INFO:teuthology.orchestra.run.smithi076.stdout:2245 read 0x5505f30 thru 0x55078e4 (0x19b5 bytes) 2024-09-14T22:53:54.835 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.835 INFO:teuthology.orchestra.run.smithi076.stdout:2249 write 0xacf441e thru 0xacfaad4 (0x66b7 bytes) 2024-09-14T22:53:54.837 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.837 INFO:teuthology.orchestra.run.smithi076.stdout:2257 read 0x7193767 thru 0x71a13aa (0xdc44 bytes) 2024-09-14T22:53:54.837 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.837 INFO:teuthology.orchestra.run.smithi076.stdout:2259 read 0xabbf75c thru 0xabc615c (0x6a01 bytes) 2024-09-14T22:53:54.837 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.837 INFO:teuthology.orchestra.run.smithi076.stdout:2261 read 0xc4fa762 thru 0xc503425 (0x8cc4 bytes) 2024-09-14T22:53:54.838 INFO:teuthology.orchestra.run.smithi076.stdout:2262 write 0xbabc545 thru 0xbabc98e (0x44a bytes) 2024-09-14T22:53:54.839 INFO:teuthology.orchestra.run.smithi076.stdout:2263 punch from 0xddda03b to 0xdde9775, (0xf73a bytes) 2024-09-14T22:53:54.839 INFO:teuthology.orchestra.run.smithi076.stdout:2264 punch from 0x24d91bb to 0x24d9ffa, (0xe3f bytes) 2024-09-14T22:53:54.839 INFO:teuthology.orchestra.run.smithi076.stdout:2265 read 0xdf57eeb thru 0xdf593d7 (0x14ed bytes) 2024-09-14T22:53:54.839 INFO:teuthology.orchestra.run.smithi076.stdout:2266 write 0xaff19e2 thru 0xafff7f2 (0xde11 bytes) 2024-09-14T22:53:54.841 INFO:teuthology.orchestra.run.smithi076.stdout:2267 read 0x5fd8069 thru 0x5fe23c8 (0xa360 bytes) 2024-09-14T22:53:54.841 INFO:teuthology.orchestra.run.smithi076.stdout:2269 writesame 0x7fc735 thru 0x806426 data_size 0x1(0x9cf2 bytes) 2024-09-14T22:53:54.843 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:54.871 INFO:teuthology.orchestra.run.smithi076.stdout:2270 writesame 0x88235e1 thru 0x88285ca data_size 0x35(0x4fea bytes) 2024-09-14T22:53:54.873 INFO:teuthology.orchestra.run.smithi076.stdout:2272 writesame 0x82010a thru 0x825ed8 data_size 0x5(0x5dcf bytes) 2024-09-14T22:53:54.877 INFO:teuthology.orchestra.run.smithi076.stdout:2274 read 0xa84570 thru 0xa8d800 (0x9291 bytes) 2024-09-14T22:53:54.878 INFO:teuthology.orchestra.run.smithi076.stdout:2275 punch from 0x4837fb5 to 0x483ce20, (0x4e6b bytes) 2024-09-14T22:53:54.878 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.878 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.878 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:54.878 INFO:teuthology.orchestra.run.smithi076.stdout:2285 write 0x42c725f thru 0x42d1202 (0x9fa4 bytes) 2024-09-14T22:53:54.879 INFO:teuthology.orchestra.run.smithi076.stdout:2286 write 0x75948fe thru 0x759acb3 (0x63b6 bytes) 2024-09-14T22:53:54.881 INFO:teuthology.orchestra.run.smithi076.stdout:2287 read 0xa4af4c thru 0xa4ee11 (0x3ec6 bytes) 2024-09-14T22:53:54.893 INFO:teuthology.orchestra.run.smithi076.stdout:2289 trunc from 0xe33f10b to 0x4b58dd5 2024-09-14T22:53:54.908 INFO:teuthology.orchestra.run.smithi076.stdout:2290 write 0x2ecc596 thru 0x2ed9019 (0xca84 bytes) 2024-09-14T22:53:54.910 INFO:teuthology.orchestra.run.smithi076.stdout:2291 read 0x125b60b thru 0x125d5c4 (0x1fba bytes) 2024-09-14T22:53:54.914 INFO:teuthology.orchestra.run.smithi076.stdout:2292 trunc from 0x4b58dd5 to 0x73c6eb9 2024-09-14T22:53:54.921 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:54.918+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:54.925 INFO:teuthology.orchestra.run.smithi076.stdout:2293 trunc from 0x73c6eb9 to 0x2b18294 2024-09-14T22:53:54.932 INFO:teuthology.orchestra.run.smithi076.stdout:2294 write 0x1bd66ef thru 0x1bdec1a (0x852c bytes) 2024-09-14T22:53:54.953 INFO:teuthology.orchestra.run.smithi076.stdout:2295 write 0xd621632 thru 0xd628621 (0x6ff0 bytes) 2024-09-14T22:53:54.954 INFO:teuthology.orchestra.run.smithi076.stdout:2296 writesame 0x5fec011 thru 0x5ff4770 data_size 0x72(0x8760 bytes) 2024-09-14T22:53:54.961 INFO:teuthology.orchestra.run.smithi076.stdout:2299 trunc from 0xd628622 to 0x9558e41 2024-09-14T22:53:54.969 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:53:54.971 INFO:teuthology.orchestra.run.smithi076.stdout:2300 read 0x1cfdb60 thru 0x1d01711 (0x3bb2 bytes) 2024-09-14T22:53:54.977 INFO:teuthology.orchestra.run.smithi076.stdout:2302 read 0x68a2a2d thru 0x68a4207 (0x17db bytes) 2024-09-14T22:53:54.977 INFO:teuthology.orchestra.run.smithi076.stdout:2303 writesame 0x39adeb8 thru 0x39b4acc data_size 0x17(0x6c15 bytes) 2024-09-14T22:53:54.980 INFO:teuthology.orchestra.run.smithi076.stdout:2304 read 0x85513aa thru 0x8553d41 (0x2998 bytes) 2024-09-14T22:53:54.980 INFO:teuthology.orchestra.run.smithi076.stdout:2306 trunc from 0x9558e41 to 0x95dfe30 2024-09-14T22:53:54.985 INFO:teuthology.orchestra.run.smithi076.stdout:2307 read 0x2dd43dd thru 0x2de0acd (0xc6f1 bytes) 2024-09-14T22:53:54.997 INFO:teuthology.orchestra.run.smithi076.stdout:2308 write 0xe90bf60 thru 0xe911d9a (0x5e3b bytes) 2024-09-14T22:53:55.002 INFO:teuthology.orchestra.run.smithi076.stdout:2309 read 0xbcba988 thru 0xbcc4267 (0x98e0 bytes) 2024-09-14T22:53:55.002 INFO:teuthology.orchestra.run.smithi076.stdout:2311 read 0xe0842d3 thru 0xe08eda9 (0xaad7 bytes) 2024-09-14T22:53:55.010 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:55.010 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:55.011 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:55.011 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:55.011 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:55.019 INFO:teuthology.orchestra.run.smithi076.stdout:2313 trunc from 0xe911d9b to 0x31d994 2024-09-14T22:53:55.021 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:55.019+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:55.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:55.038 INFO:teuthology.orchestra.run.smithi076.stdout:2314 punch from 0xe42a to 0x1b8eb, (0xd4c1 bytes) 2024-09-14T22:53:55.038 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:55.053 INFO:teuthology.orchestra.run.smithi076.stdout:2317 write 0x8f8ec51 thru 0x8f9e3f9 (0xf7a9 bytes) 2024-09-14T22:53:55.055 INFO:teuthology.orchestra.run.smithi076.stdout:2318 read 0x2668dbd thru 0x26701eb (0x742f bytes) 2024-09-14T22:53:55.064 INFO:teuthology.orchestra.run.smithi076.stdout:2319 writesame 0xc7bc337 thru 0xc7c3fcd data_size 0x1(0x7c97 bytes) 2024-09-14T22:53:55.066 INFO:teuthology.orchestra.run.smithi076.stdout:2320 read 0x75a062a thru 0x75a5777 (0x514e bytes) 2024-09-14T22:53:55.066 INFO:teuthology.orchestra.run.smithi076.stdout:2324 read 0x903928b thru 0x9041e1b (0x8b91 bytes) 2024-09-14T22:53:55.066 INFO:teuthology.orchestra.run.smithi076.stdout:2325 punch from 0x50b3551 to 0x50b5bfe, (0x26ad bytes) 2024-09-14T22:53:55.066 INFO:teuthology.orchestra.run.smithi076.stdout:2326 punch from 0x447108e to 0x4478a32, (0x79a4 bytes) 2024-09-14T22:53:55.066 INFO:teuthology.orchestra.run.smithi076.stdout:2327 read 0xf29105 thru 0xf346a4 (0xb5a0 bytes) 2024-09-14T22:53:55.069 INFO:teuthology.orchestra.run.smithi076.stdout:2328 trunc from 0xc7c3fce to 0x9f38b13 2024-09-14T22:53:55.075 INFO:teuthology.orchestra.run.smithi076.stdout:2329 read 0x502626 thru 0x50b7b3 (0x918e bytes) 2024-09-14T22:53:55.076 INFO:teuthology.orchestra.run.smithi076.stdout:2330 writesame 0x16c7ef4 thru 0x16ccb55 data_size 0x3(0x4c62 bytes) 2024-09-14T22:53:55.082 INFO:teuthology.orchestra.run.smithi076.stdout:2331 punch from 0x35431ec to 0x3544d1d, (0x1b31 bytes) 2024-09-14T22:53:55.084 INFO:teuthology.orchestra.run.smithi076.stdout:2332 trunc from 0x9f38b13 to 0x8aa68c7 2024-09-14T22:53:55.098 INFO:teuthology.orchestra.run.smithi076.stdout:2333 writesame 0xa1827d8 thru 0xa185c08 data_size 0x1(0x3431 bytes) 2024-09-14T22:53:55.098 INFO:teuthology.orchestra.run.smithi076.stdout:2334 punch from 0x82faba2 to 0x82fbd4c, (0x11aa bytes) 2024-09-14T22:53:55.099 INFO:teuthology.orchestra.run.smithi076.stdout:2335 read 0x91ec3a3 thru 0x91eec8c (0x28ea bytes) 2024-09-14T22:53:55.099 INFO:teuthology.orchestra.run.smithi076.stdout:2336 write 0x33e8e6b thru 0x33e8fb4 (0x14a bytes) 2024-09-14T22:53:55.105 INFO:teuthology.orchestra.run.smithi076.stdout:2338 trunc from 0xa185c09 to 0x5dba3d1 2024-09-14T22:53:55.112 INFO:teuthology.orchestra.run.smithi076.stdout:2339 write 0x1779737 thru 0x178396d (0xa237 bytes) 2024-09-14T22:53:55.113 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:55.122 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:55.120+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:55.123 INFO:teuthology.orchestra.run.smithi076.stdout:2341 writesame 0x94317b0 thru 0x9431d1e data_size 0x1(0x56f bytes) 2024-09-14T22:53:55.124 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:55.126 INFO:teuthology.orchestra.run.smithi076.stdout:2343 trunc from 0x9431d1f to 0xae8ac20 2024-09-14T22:53:55.131 INFO:teuthology.orchestra.run.smithi076.stdout:2344 write 0x79d48aa thru 0x79ddc79 (0x93d0 bytes) 2024-09-14T22:53:55.132 INFO:teuthology.orchestra.run.smithi076.stdout:2345 write 0x481fcf2 thru 0x4823cd4 (0x3fe3 bytes) 2024-09-14T22:53:55.134 INFO:teuthology.orchestra.run.smithi076.stdout:2346 read 0x7541b6c thru 0x75431d3 (0x1668 bytes) 2024-09-14T22:53:55.134 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:55.134 INFO:teuthology.orchestra.run.smithi076.stdout:2349 punch from 0x912877d to 0x9137e0a, (0xf68d bytes) 2024-09-14T22:53:55.134 INFO:teuthology.orchestra.run.smithi076.stdout:2350 write 0x5efe591 thru 0x5f0bd64 (0xd7d4 bytes) 2024-09-14T22:53:55.140 INFO:teuthology.orchestra.run.smithi076.stdout:2351 read 0x6e21615 thru 0x6e30d53 (0xf73f bytes) 2024-09-14T22:53:55.140 INFO:teuthology.orchestra.run.smithi076.stdout:2352 read 0x805077 thru 0x8133bd (0xe347 bytes) 2024-09-14T22:53:55.140 INFO:teuthology.orchestra.run.smithi076.stdout:2353 read 0x6cd216c thru 0x6cdb701 (0x9596 bytes) 2024-09-14T22:53:55.140 INFO:teuthology.orchestra.run.smithi076.stdout:2354 write 0x777bc26 thru 0x77896c9 (0xdaa4 bytes) 2024-09-14T22:53:55.143 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:55.143 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:55.143 INFO:teuthology.orchestra.run.smithi076.stdout:2359 read 0x645d873 thru 0x646739e (0x9b2c bytes) 2024-09-14T22:53:55.147 INFO:teuthology.orchestra.run.smithi076.stdout:2360 trunc from 0xae8ac20 to 0xd3c1031 2024-09-14T22:53:55.152 INFO:teuthology.orchestra.run.smithi076.stdout:2362 clone 22 order 21 su 1048576 sc 3 2024-09-14T22:53:55.166 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.168315342, 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.168296303999998, 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:53:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.168260348, 2024-09-14T22:53:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:53:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.168239228000001, 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:53:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:53:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.167032314, 2024-09-14T22:53:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.165202867000001, 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:53:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:53:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.164648580000001, 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:53:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.162663861999999, 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:53:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:53:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:53:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:53:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.162311542000001, 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.162028099, 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.161740001999998, 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:53:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:53:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.161365174, 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.161044359000002, 2024-09-14T22:53:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:53:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:53:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.147359124000001, 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.147024314999999, 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:53:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.146694152999999, 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:53:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:53:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.146473366999999, 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:53:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.146204065999999, 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:53:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:53:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.146016970000002, 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.145698810999999, 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:53:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.252 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:55.254 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:55.254 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:55.252+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:55.354 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:55.353+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:55.398 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:55.398 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:55.398 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:55.398 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:55.406 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:55.455 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:55.453+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:55.549 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:55.549 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:55.549 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:55.549 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:55.549 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:55.556 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:55.554+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:55.558 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:55.657 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:55.655+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8992 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'23, mlcod=33'21)", 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.754469280000002, 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0035453559999999999, 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.663 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906376+0000", 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.42e-06 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906968+0000", 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00059252399999999998 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906990+0000", 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1498999999999999e-05 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906999+0000", 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2299999999999997e-06 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907058+0000", 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9505e-05 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907082+0000", 2024-09-14T22:53:55.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3298999999999998e-05 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909874+0000", 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027926689999999998 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909919+0000", 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5210000000000003e-05 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8998 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'25, mlcod=33'21)", 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.752801345999998, 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003268272, 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:55.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908043+0000", 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908054+0000", 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1654999999999999e-05 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908066+0000", 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1321000000000001e-05 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908077+0000", 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1781e-05 2024-09-14T22:53:55.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908149+0000", 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1402000000000005e-05 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908170+0000", 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0746000000000001e-05 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911259+0000", 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030889839999999999 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911310+0000", 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1684999999999997e-05 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15153 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'58, mlcod=34'56)", 2024-09-14T22:53:55.672 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 36.950626927999998, 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043627650000000002, 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:55.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710218+0000", 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.291e-06 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710957+0000", 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00073905599999999998 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710977+0000", 2024-09-14T22:53:55.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0506999999999999e-05 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710986+0000", 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0860000000000008e-06 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711048+0000", 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1130999999999995e-05 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711072+0000", 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4167000000000001e-05 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714533+0000", 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034611070000000002 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714579+0000", 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6419999999999999e-05 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15154 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'60, mlcod=34'56)", 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 36.949276173000001, 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00342618, 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711568+0000", 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0900000000000002e-07 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711574+0000", 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5119999999999997e-06 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711583+0000", 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7579999999999998e-06 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711589+0000", 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9619999999999998e-06 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711692+0000", 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000103237 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711719+0000", 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7124999999999999e-05 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714953+0000", 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032341240000000001 2024-09-14T22:53:55.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714993+0000", 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9953000000000002e-05 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77339 3.9 e40/24 3:939e5cf2:::rbd_data.113564f98e71.0000000000000044:head v 40'60, mlcod=40'58)", 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.628660406000002, 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0019342230000000001, 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.681 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032184+0000", 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032626+0000", 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000441369 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032644+0000", 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8029999999999998e-05 2024-09-14T22:53:55.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032652+0000", 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7760000000000003e-06 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032720+0000", 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032743+0000", 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2823000000000002e-05 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034080+0000", 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001337705 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034117+0000", 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6854000000000003e-05 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.685 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77340 3.9 e40/24 3:939e5cf2:::rbd_data.113564f98e71.0000000000000044:head v 40'62, mlcod=40'58)", 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.627771364000001, 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00147386, 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033073+0000", 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9999999999999998e-07 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033078+0000", 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5430000000000001e-06 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033087+0000", 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5420000000000005e-06 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033093+0000", 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9479999999999996e-06 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033185+0000", 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2298000000000003e-05 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033218+0000", 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3389999999999997e-05 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034501+0000", 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012830160000000001 2024-09-14T22:53:55.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034546+0000", 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4623000000000003e-05 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.452321934, 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:53:55.690 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:53:55.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:53:55.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.694 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:78175 3.4 e41/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 41'105, mlcod=40'102)", 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.69504298, 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0049595489999999997, 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965801+0000", 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.35e-07 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965834+0000", 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3683999999999998e-05 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965846+0000", 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1620999999999999e-05 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965855+0000", 2024-09-14T22:53:55.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1130000000000007e-06 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965939+0000", 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4029000000000005e-05 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965963+0000", 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3529000000000001e-05 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970705+0000", 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.004742853 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970760+0000", 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4484999999999998e-05 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:116229 3.1e e48/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 48'99, mlcod=48'97)", 2024-09-14T22:53:55.699 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.356751078999999, 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034125309999999999, 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304095+0000", 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3029999999999998e-06 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304126+0000", 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1374999999999997e-05 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304140+0000", 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4518000000000001e-05 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304150+0000", 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304213+0000", 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3366000000000003e-05 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304237+0000", 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4379000000000002e-05 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307471+0000", 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032338829999999999 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307505+0000", 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3552e-05 2024-09-14T22:53:55.703 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 15.32866016, 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:55.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:131796 3.1e e53/44 3:79e10459:::rbd_data.1135a5fb387.0000000000000058:head v 53'145, mlcod=52'143)", 2024-09-14T22:53:55.708 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.099396484, 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012088559999999999, 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8e-07 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561522+0000", 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4567999999999995e-05 2024-09-14T22:53:55.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561534+0000", 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2517999999999999e-05 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561546+0000", 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1440000000000001e-05 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561630+0000", 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4615000000000002e-05 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561659+0000", 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8969e-05 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.562606+0000", 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00094671999999999996 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.562656+0000", 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9845999999999997e-05 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:133036 3.4 e54/24 3:22b239af:::rbd_data.1135a4365f9c.0000000000000001:head v 54'137, mlcod=53'135)", 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 13.390960210999999, 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002821278, 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.713 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7100000000000001e-07 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270380+0000", 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00049641500000000005 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270393+0000", 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3088000000000001e-05 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270402+0000", 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6199999999999994e-06 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270470+0000", 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7459999999999994e-05 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270498+0000", 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8214000000000001e-05 2024-09-14T22:53:55.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272657+0000", 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021591499999999999 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272704+0000", 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7160000000000002e-05 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:135238 3.4 e56/24 3:2474bdb6:::rbd_object_map.113530dd5d95:head v 56'311, mlcod=56'309)", 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.034818+0000", 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.626025201999999, 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0015748439999999999, 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034818+0000", 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034818+0000", 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034819+0000", 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034829+0000", 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0139999999999999e-05 2024-09-14T22:53:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034844+0000", 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4620999999999999e-05 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034852+0000", 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.909e-06 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034904+0000", 2024-09-14T22:53:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2080000000000003e-05 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034924+0000", 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0227000000000001e-05 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.036344+0000", 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0014203219999999999 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.036393+0000", 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8662000000000003e-05 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 6.3968926890000004, 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:53:55.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.726 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:138018 3.9 e61/24 3:94720cbd:::rbd_data.1135d9b82239.000000000000003f:head v 61'108, mlcod=60'106)", 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:50.257911+0000", 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.4029321069999998, 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001337092, 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257911+0000", 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257911+0000", 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257912+0000", 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2999999999999999e-07 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257920+0000", 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8169999999999999e-06 2024-09-14T22:53:55.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257931+0000", 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.101e-05 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257937+0000", 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6470000000000003e-06 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257989+0000", 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1929999999999999e-05 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.258011+0000", 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2218e-05 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.259210+0000", 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001198787 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.259248+0000", 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8753e-05 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:138416 3.5 e61/47 3:a3cf4870:::rbd_data.1135d9b82239.0000000000000013:head v 61'157, mlcod=60'155)", 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:50.401811+0000", 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.2590320430000004, 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0022471959999999999, 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.731 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401811+0000", 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401811+0000", 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401813+0000", 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2750000000000001e-06 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401824+0000", 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1255e-05 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401837+0000", 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2988e-05 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401843+0000", 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1580000000000002e-06 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401893+0000", 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9963000000000003e-05 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401914+0000", 2024-09-14T22:53:55.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1007000000000001e-05 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.404020+0000", 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002105787 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.404059+0000", 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8763000000000001e-05 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:140152 3.5 e61/47 3:a3cf4870:::rbd_data.1135d9b82239.0000000000000013:head v 61'163, mlcod=61'161)", 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:50.810788+0000", 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.8500557329999996, 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021170479999999998, 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810788+0000", 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810788+0000", 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810789+0000", 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1030000000000001e-06 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810834+0000", 2024-09-14T22:53:55.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5629e-05 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810849+0000", 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4705999999999999e-05 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810856+0000", 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5230000000000001e-06 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810906+0000", 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9997000000000003e-05 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810922+0000", 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6274999999999999e-05 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.812871+0000", 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0019489640000000001 2024-09-14T22:53:55.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.812905+0000", 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3850999999999998e-05 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:141657 3.1e e65/44 3:7b172f22:::rbd_data.113580c09eae.000000000000000c:head v 65'217, mlcod=64'214)", 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:54.024381+0000", 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.6364626979999999, 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00127625, 2024-09-14T22:53:55.740 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024381+0000", 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024381+0000", 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024381+0000", 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024386+0000", 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2499999999999997e-06 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024397+0000", 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0564000000000001e-05 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024410+0000", 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3563999999999999e-05 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024467+0000", 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7074000000000003e-05 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024488+0000", 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0203999999999998e-05 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.025632+0000", 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011444999999999999 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.025657+0000", 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4749999999999999e-05 2024-09-14T22:53:55.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:141677 3.1e e65/44 3:7e4db642:::rbd_data.113580c09eae.0000000000000007:head v 65'219, mlcod=64'215)", 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:54.037906+0000", 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.6229376179999999, 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001637555, 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037906+0000", 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037906+0000", 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037907+0000", 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.186e-06 2024-09-14T22:53:55.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037917+0000", 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0156999999999999e-05 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037933+0000", 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6322999999999999e-05 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037942+0000", 2024-09-14T22:53:55.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0280000000000006e-06 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.038000+0000", 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7816999999999999e-05 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.038023+0000", 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2687000000000001e-05 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.039514+0000", 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001490812 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.039543+0000", 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9544999999999999e-05 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:142014 3.1e e65/44 3:7ae60eba:::rbd_data.1135a7a7c51e.0000000000000000:head v 65'220, mlcod=65'217)", 2024-09-14T22:53:55.749 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.4708894029999999, 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0042481029999999996, 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:53:55.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189955+0000", 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0979999999999999e-06 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191197+0000", 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012418710000000001 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:55.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191217+0000", 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.978e-05 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191226+0000", 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6500000000000008e-06 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191288+0000", 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1227000000000002e-05 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191312+0000", 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4088999999999999e-05 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.194153+0000", 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002841437 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:55.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.194202+0000", 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8950999999999999e-05 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:55.754 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:55.755 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:55.758 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:55.757+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:55.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:53:55.859 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:55.857+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:55.859 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:55.859 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:55.859 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:55.859 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:55.868 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:55.959 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:53:55.959 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:53:55.959 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-14T22:53:55.959 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T22:53:55.959 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T22:53:55.960 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-14T22:53:55.961 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:55.958+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:55.982 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:55.982 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:56.002 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:56.002 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:56.018 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:56.019 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:56.023 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:56.023 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:56.023 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:56.023 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:56.024 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:56.033 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:56.037 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:56.037 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:56.054 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone21 from 0xd3c1031 (overlap 0x116c4d) to 0xdf03d 2024-09-14T22:53:56.057 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:56.057 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:56.061 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:56.060+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:56.077 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:56.077 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:56.081 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:56.081 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T22:53:56.081 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:56.095 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:56.096 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:56.117 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:56.117 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:56.137 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T22:53:56.151 INFO:teuthology.orchestra.run.smithi076.stdout:2364 read 0x4545253 thru 0x454914c (0x3efa bytes) 2024-09-14T22:53:56.155 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:56.155 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:56.160 INFO:teuthology.orchestra.run.smithi076.stdout:2365 trunc from 0xd3c1031 to 0x74813a8 2024-09-14T22:53:56.162 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:56.159+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:56.167 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:56.172 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:56.172 INFO:teuthology.orchestra.run.smithi076.stdout:2367 punch from 0x4fed356 to 0x4ff1d51, (0x49fb bytes) 2024-09-14T22:53:56.172 INFO:teuthology.orchestra.run.smithi076.stdout:2369 writesame 0x30f9ce6 thru 0x30fa1aa data_size 0x3(0x4c5 bytes) 2024-09-14T22:53:56.190 INFO:teuthology.orchestra.run.smithi076.stdout:2370 write 0xc800028 thru 0xc80cb09 (0xcae2 bytes) 2024-09-14T22:53:56.193 INFO:teuthology.orchestra.run.smithi076.stdout:2371 write 0x6cf71fa thru 0x6cfc099 (0x4ea0 bytes) 2024-09-14T22:53:56.197 INFO:teuthology.orchestra.run.smithi076.stdout:2372 write 0x8e5539d thru 0x8e5b8f1 (0x6555 bytes) 2024-09-14T22:53:56.200 INFO:teuthology.orchestra.run.smithi076.stdout:2373 write 0x31a1998 thru 0x31a1a08 (0x71 bytes) 2024-09-14T22:53:56.203 INFO:teuthology.orchestra.run.smithi076.stdout:2374 punch from 0x60fd6f3 to 0x610c860, (0xf16d bytes) 2024-09-14T22:53:56.203 INFO:teuthology.orchestra.run.smithi076.stdout:2376 punch from 0xa2ea122 to 0xa2f17b4, (0x7692 bytes) 2024-09-14T22:53:56.203 INFO:teuthology.orchestra.run.smithi076.stdout:2378 write 0xb5c746c thru 0xb5d2989 (0xb51e bytes) 2024-09-14T22:53:56.214 INFO:teuthology.orchestra.run.smithi076.stdout:2379 trunc from 0xc80cb0a to 0x5b4d279 2024-09-14T22:53:56.229 INFO:teuthology.orchestra.run.smithi076.stdout:2380 read 0x293048e thru 0x2938ea7 (0x8a1a bytes) 2024-09-14T22:53:56.237 INFO:teuthology.orchestra.run.smithi076.stdout:2382 write 0x751ecdc thru 0x7527a05 (0x8d2a bytes) 2024-09-14T22:53:56.240 INFO:teuthology.orchestra.run.smithi076.stdout:2383 punch from 0x1d92894 to 0x1d9473d, (0x1ea9 bytes) 2024-09-14T22:53:56.241 INFO:teuthology.orchestra.run.smithi076.stdout:2384 read 0x16af0c4 thru 0x16b5af9 (0x6a36 bytes) 2024-09-14T22:53:56.241 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:56.241 INFO:teuthology.orchestra.run.smithi076.stdout:2388 read 0x4cc3ae5 thru 0x4cc9e99 (0x63b5 bytes) 2024-09-14T22:53:56.241 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:56.242 INFO:teuthology.orchestra.run.smithi076.stdout:2391 write 0x4379b4e thru 0x4386974 (0xce27 bytes) 2024-09-14T22:53:56.245 INFO:teuthology.orchestra.run.smithi076.stdout:2392 read 0x41557ed thru 0x415b03b (0x584f bytes) 2024-09-14T22:53:56.245 INFO:teuthology.orchestra.run.smithi076.stdout:2393 read 0x32f8696 thru 0x3300da1 (0x870c bytes) 2024-09-14T22:53:56.251 INFO:teuthology.orchestra.run.smithi076.stdout:2395 trunc from 0x7527a06 to 0x2c14c30 2024-09-14T22:53:56.262 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:56.260+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:56.271 INFO:teuthology.orchestra.run.smithi076.stdout:2396 read 0x1bbe1f1 thru 0x1bc5604 (0x7414 bytes) 2024-09-14T22:53:56.281 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:56.282 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:56.288 INFO:teuthology.orchestra.run.smithi076.stdout:2398 write 0xbfa2883 thru 0xbfa6a21 (0x419f bytes) 2024-09-14T22:53:56.291 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:56.292 INFO:teuthology.orchestra.run.smithi076.stdout:2399 punch from 0x4217393 to 0x421970d, (0x237a bytes) 2024-09-14T22:53:56.292 INFO:teuthology.orchestra.run.smithi076.stdout:2400 write 0xc0a502 thru 0xc15c02 (0xb701 bytes) 2024-09-14T22:53:56.295 INFO:teuthology.orchestra.run.smithi076.stdout:2401 write 0x15adb40 thru 0x15adfd5 (0x496 bytes) 2024-09-14T22:53:56.298 INFO:teuthology.orchestra.run.smithi076.stdout:2402 clone 23 order 20 su 4096 sc 7 2024-09-14T22:53:56.364 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:56.362+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:56.434 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:56.435 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:56.443 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:53:56.464 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:56.463+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:56.545 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:56.545 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:56.553 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:53:56.565 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:56.564+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:56.655 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:56.656 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:56.664 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:53:56.666 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:56.664+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:56.767 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:56.767 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:56.767 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:56.765+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:56.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:53:56.868 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:56.865+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:56.918 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:56.918 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:56.927 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:53:56.968 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:56.967+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:56.988 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:53:57.067 INFO:teuthology.orchestra.run.smithi076.stdout:flattening image image_client.0-clone22 2024-09-14T22:53:57.069 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:57.068+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:57.080 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:57.080 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:57.089 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:53:57.170 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:57.168+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:57.192 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:57.193 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:57.202 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:53:57.238 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T22:53:57.243 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:57.243 INFO:teuthology.orchestra.run.smithi076.stdout:2404 write 0x1665e8d thru 0x1674d2a (0xee9e bytes) 2024-09-14T22:53:57.287 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:57.285+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:57.296 INFO:teuthology.orchestra.run.smithi076.stdout:2406 trunc from 0xbfa6a22 to 0xc8718b4 2024-09-14T22:53:57.301 INFO:teuthology.orchestra.run.smithi076.stdout:2407 punch from 0x7286d79 to 0x7287a7d, (0xd04 bytes) 2024-09-14T22:53:57.301 INFO:teuthology.orchestra.run.smithi076.stdout:2409 punch from 0x54f4253 to 0x54f7983, (0x3730 bytes) 2024-09-14T22:53:57.302 INFO:teuthology.orchestra.run.smithi076.stdout:2412 read 0x9e3b321 thru 0x9e4254c (0x722c bytes) 2024-09-14T22:53:57.302 INFO:teuthology.orchestra.run.smithi076.stdout:2414 write 0x641330a thru 0x6419e9d (0x6b94 bytes) 2024-09-14T22:53:57.311 INFO:teuthology.orchestra.run.smithi076.stdout:2415 punch from 0x4952238 to 0x4954f7c, (0x2d44 bytes) 2024-09-14T22:53:57.311 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:57.313 INFO:teuthology.orchestra.run.smithi076.stdout:2417 trunc from 0xc8718b4 to 0xa2ee3f3 2024-09-14T22:53:57.327 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:57.327 INFO:teuthology.orchestra.run.smithi076.stdout:2423 clone 24 order 25 su 131072 sc 7 2024-09-14T22:53:57.361 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:57.361 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:57.370 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:53:57.372 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:57.371+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:57.473 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:57.472+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:57.511 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:57.511 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:57.520 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:53:57.574 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:57.573+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:57.662 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:57.662 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:57.671 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:53:57.675 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:57.674+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:57.774 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:57.774 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:57.775 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:57.774+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:57.783 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:53:57.876 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:57.874+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:57.891 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:57.892 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:57.900 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:53:57.976 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:57.975+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:57.984 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:53:58.044 INFO:teuthology.orchestra.run.smithi076.stdout:flattening image image_client.0-clone23 2024-09-14T22:53:58.064 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:58.064 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:58.074 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:53:58.078 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:58.076+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:58.130 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:53:58.179 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:58.177+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:58.209 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:53:58.209 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:53:58.218 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:53:58.271 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:53:58.271 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:53:58.279 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:58.278+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:58.289 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:53:58.290 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:53:58.306 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:53:58.307 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:53:58.324 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:53:58.324 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:53:58.337 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T22:53:58.341 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:53:58.341 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:53:58.358 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:53:58.359 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:53:58.371 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:58.372 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:58.372 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:58.372 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:58.376 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:53:58.377 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:53:58.380 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:58.379+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:58.384 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:53:58.396 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:53:58.396 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:53:58.481 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:58.480+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:58.508 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:58.508 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:58.508 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:58.508 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:58.508 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:58.517 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:53:58.582 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:53:58.580+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:53:58.595 INFO:teuthology.orchestra.run.smithi076.stdout:2425 read 0x70e4ca3 thru 0x70e918e (0x44ec bytes) 2024-09-14T22:53:58.595 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.595 INFO:teuthology.orchestra.run.smithi076.stdout:2428 writesame 0x87c26c3 thru 0x87c591f data_size 0x1(0x325d bytes) 2024-09-14T22:53:58.624 INFO:teuthology.orchestra.run.smithi076.stdout:2430 write 0xa60e313 thru 0xa61d908 (0xf5f6 bytes) 2024-09-14T22:53:58.633 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:58.633 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.634968773000001, 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.634 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:53:58.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:58.638 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.634949734999999, 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:58.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:53:58.642 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.634913779000001, 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.643 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:53:58.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:58.647 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.634892658999998, 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:58.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:53:58.651 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.633685745000001, 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.652 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:53:58.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:53:58.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.656 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.631856297999999, 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:53:58.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.658 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.631302010999999, 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.661 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:53:58.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.629317293, 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:53:58.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:53:58.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.628964972999999, 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.670 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:53:58.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.674 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.628681530000001, 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.675 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:53:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.628393432999999, 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.679 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:53:58.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.681 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.683 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.628018605000001, 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:53:58.687 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.627697789999999, 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.688 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:53:58.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:53:58.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.614012554999999, 2024-09-14T22:53:58.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:53:58.696 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.613677746, 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.697 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:53:58.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.613347584, 2024-09-14T22:53:58.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:53:58.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.613126798, 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:53:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.612857497, 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:53:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:53:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.612670400999999, 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:53:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 21.612352242, 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:53:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:53:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:53:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:58.723 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:58.725 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:53:58.726 INFO:teuthology.orchestra.run.smithi076.stdout:2431 read 0x51607b3 thru 0x516bf77 (0xb7c5 bytes) 2024-09-14T22:53:58.726 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.726 INFO:teuthology.orchestra.run.smithi076.stdout:2433 write 0x3ce1560 thru 0x3ceddf8 (0xc899 bytes) 2024-09-14T22:53:58.726 INFO:teuthology.orchestra.run.smithi076.stdout:2434 read 0x474b84 thru 0x47d126 (0x85a3 bytes) 2024-09-14T22:53:58.726 INFO:teuthology.orchestra.run.smithi076.stdout:2437 punch from 0x41b2193 to 0x41b8a2d, (0x689a bytes) 2024-09-14T22:53:58.726 INFO:teuthology.orchestra.run.smithi076.stdout:2438 trunc from 0xa61d909 to 0xc12fdb3 2024-09-14T22:53:58.726 INFO:teuthology.orchestra.run.smithi076.stdout:2439 write 0x79fc378 thru 0x7a091aa (0xce33 bytes) 2024-09-14T22:53:58.727 INFO:teuthology.orchestra.run.smithi076.stdout:2440 write 0x53010e2 thru 0x5309a59 (0x8978 bytes) 2024-09-14T22:53:58.727 INFO:teuthology.orchestra.run.smithi076.stdout:2442 read 0x1bcd98e thru 0x1bd4df8 (0x746b bytes) 2024-09-14T22:53:58.727 INFO:teuthology.orchestra.run.smithi076.stdout:2443 write 0x91c8e25 thru 0x91d6044 (0xd220 bytes) 2024-09-14T22:53:58.727 INFO:teuthology.orchestra.run.smithi076.stdout:2445 read 0x5288030 thru 0x528b0dc (0x30ad bytes) 2024-09-14T22:53:58.727 INFO:teuthology.orchestra.run.smithi076.stdout:2446 read 0x917fa14 thru 0x918f4fd (0xfaea bytes) 2024-09-14T22:53:58.727 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.727 INFO:teuthology.orchestra.run.smithi076.stdout:2448 writesame 0x8bce585 thru 0x8bcfae5 data_size 0x1(0x1561 bytes) 2024-09-14T22:53:58.728 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:58.723+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:58.728 INFO:teuthology.orchestra.run.smithi076.stdout:2449 writesame 0xe8b6d12 thru 0xe8c266b data_size 0xd(0xb95a bytes) 2024-09-14T22:53:58.728 INFO:teuthology.orchestra.run.smithi076.stdout:2451 writesame 0x691b57b thru 0x6924ba2 data_size 0x28(0x9628 bytes) 2024-09-14T22:53:58.750 INFO:teuthology.orchestra.run.smithi076.stdout:2452 trunc from 0xe8c266c to 0xae15f2d 2024-09-14T22:53:58.766 INFO:teuthology.orchestra.run.smithi076.stdout:2454 writesame 0xdf857f9 thru 0xdf8d164 data_size 0x2(0x796c bytes) 2024-09-14T22:53:58.767 INFO:teuthology.orchestra.run.smithi076.stdout:2455 write 0x3aa31a2 thru 0x3aa9c40 (0x6a9f bytes) 2024-09-14T22:53:58.768 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.769 INFO:teuthology.orchestra.run.smithi076.stdout:2457 write 0x28ddfba thru 0x28ed711 (0xf758 bytes) 2024-09-14T22:53:58.770 INFO:teuthology.orchestra.run.smithi076.stdout:2459 trunc from 0xdf8d165 to 0xde613a2 2024-09-14T22:53:58.776 INFO:teuthology.orchestra.run.smithi076.stdout:2460 read 0xbce0af9 thru 0xbce85d2 (0x7ada bytes) 2024-09-14T22:53:58.776 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.777 INFO:teuthology.orchestra.run.smithi076.stdout:2462 read 0x57edf69 thru 0x57fb441 (0xd4d9 bytes) 2024-09-14T22:53:58.777 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.777 INFO:teuthology.orchestra.run.smithi076.stdout:2464 read 0x45cb086 thru 0x45d8369 (0xd2e4 bytes) 2024-09-14T22:53:58.777 INFO:teuthology.orchestra.run.smithi076.stdout:2465 writesame 0x50341f1 thru 0x5043399 data_size 0x1(0xf1a9 bytes) 2024-09-14T22:53:58.819 INFO:teuthology.orchestra.run.smithi076.stdout:2466 write 0x7b1bed2 thru 0x7b1f519 (0x3648 bytes) 2024-09-14T22:53:58.822 INFO:teuthology.orchestra.run.smithi076.stdout:2467 write 0x7fae106 thru 0x7fba8bd (0xc7b8 bytes) 2024-09-14T22:53:58.824 INFO:teuthology.orchestra.run.smithi076.stdout:2468 trunc from 0xde613a2 to 0xe0ad3d7 2024-09-14T22:53:58.825 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:58.823+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:58.829 INFO:teuthology.orchestra.run.smithi076.stdout:2469 read 0x3386e2a thru 0x3395455 (0xe62c bytes) 2024-09-14T22:53:58.830 INFO:teuthology.orchestra.run.smithi076.stdout:2470 write 0x6768d87 thru 0x6776588 (0xd802 bytes) 2024-09-14T22:53:58.832 INFO:teuthology.orchestra.run.smithi076.stdout:2471 write 0x95bffd5 thru 0x95ceaa7 (0xead3 bytes) 2024-09-14T22:53:58.833 INFO:teuthology.orchestra.run.smithi076.stdout:2472 write 0x13471f2 thru 0x135158e (0xa39d bytes) 2024-09-14T22:53:58.835 INFO:teuthology.orchestra.run.smithi076.stdout:2473 write 0xddfc012 thru 0xde03cce (0x7cbd bytes) 2024-09-14T22:53:58.836 INFO:teuthology.orchestra.run.smithi076.stdout:2474 read 0xbda64d1 thru 0xbda66d9 (0x209 bytes) 2024-09-14T22:53:58.836 INFO:teuthology.orchestra.run.smithi076.stdout:2476 write 0x863366d thru 0x863ad89 (0x771d bytes) 2024-09-14T22:53:58.837 INFO:teuthology.orchestra.run.smithi076.stdout:2477 read 0x54cd917 thru 0x54db55f (0xdc49 bytes) 2024-09-14T22:53:58.838 INFO:teuthology.orchestra.run.smithi076.stdout:2478 writesame 0x4f2744f thru 0x4f34b12 data_size 0xa(0xd6c4 bytes) 2024-09-14T22:53:58.843 INFO:teuthology.orchestra.run.smithi076.stdout:2480 punch from 0x1616b0a to 0x161cbe3, (0x60d9 bytes) 2024-09-14T22:53:58.843 INFO:teuthology.orchestra.run.smithi076.stdout:2481 read 0x83b3894 thru 0x83be881 (0xafee bytes) 2024-09-14T22:53:58.843 INFO:teuthology.orchestra.run.smithi076.stdout:2482 write 0x92b10ea thru 0x92bf438 (0xe34f bytes) 2024-09-14T22:53:58.845 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:58.845 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:58.845 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:58.845 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:58.845 INFO:teuthology.orchestra.run.smithi076.stdout:2484 read 0x234a612 thru 0x2358107 (0xdaf6 bytes) 2024-09-14T22:53:58.845 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.855 INFO:teuthology.orchestra.run.smithi076.stdout:2486 trunc from 0xe0ad3d7 to 0x70eadcc 2024-09-14T22:53:58.863 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:53:58.867 INFO:teuthology.orchestra.run.smithi076.stdout:2488 write 0x29ee0bb thru 0x29f2f5c (0x4ea2 bytes) 2024-09-14T22:53:58.869 INFO:teuthology.orchestra.run.smithi076.stdout:2489 read 0x5e1ca6d thru 0x5e29f93 (0xd527 bytes) 2024-09-14T22:53:58.870 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.870 INFO:teuthology.orchestra.run.smithi076.stdout:2491 writesame 0x3a87ab7 thru 0x3a8effb data_size 0x1(0x7545 bytes) 2024-09-14T22:53:58.888 INFO:teuthology.orchestra.run.smithi076.stdout:2492 write 0x3851922 thru 0x3858bc0 (0x729f bytes) 2024-09-14T22:53:58.890 INFO:teuthology.orchestra.run.smithi076.stdout:2494 punch from 0x18e26dc to 0x18f115c, (0xea80 bytes) 2024-09-14T22:53:58.890 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.894 INFO:teuthology.orchestra.run.smithi076.stdout:2497 trunc from 0x70eadcc to 0x3b2b588 2024-09-14T22:53:58.920 INFO:teuthology.orchestra.run.smithi076.stdout:2499 writesame 0xc151f8f thru 0xc1617f3 data_size 0x1(0xf865 bytes) 2024-09-14T22:53:58.921 INFO:teuthology.orchestra.run.smithi076.stdout:2500 read 0x84a315a thru 0x84a443c (0x12e3 bytes) 2024-09-14T22:53:58.924 INFO:teuthology.orchestra.run.smithi076.stdout:2501 trunc from 0xc1617f4 to 0x9ca6308 2024-09-14T22:53:58.925 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:53:58.924+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:53:58.931 INFO:teuthology.orchestra.run.smithi076.stdout:2502 read 0x2a87aa1 thru 0x2a95e50 (0xe3b0 bytes) 2024-09-14T22:53:58.940 INFO:teuthology.orchestra.run.smithi076.stdout:2503 write 0xd6a5482 thru 0xd6b265b (0xd1da bytes) 2024-09-14T22:53:58.942 INFO:teuthology.orchestra.run.smithi076.stdout:2505 write 0x4a74af3 thru 0x4a793db (0x48e9 bytes) 2024-09-14T22:53:58.944 INFO:teuthology.orchestra.run.smithi076.stdout:2507 read 0x4318f0a thru 0x4323fc1 (0xb0b8 bytes) 2024-09-14T22:53:58.945 INFO:teuthology.orchestra.run.smithi076.stdout:2508 read 0x8ab200e thru 0x8ac1cec (0xfcdf bytes) 2024-09-14T22:53:58.945 INFO:teuthology.orchestra.run.smithi076.stdout:2509 punch from 0xc5f64c1 to 0xc5fc6b6, (0x61f5 bytes) 2024-09-14T22:53:58.945 INFO:teuthology.orchestra.run.smithi076.stdout:2510 writesame 0xc9dacb6 thru 0xc9e401a data_size 0x1(0x9365 bytes) 2024-09-14T22:53:58.977 INFO:teuthology.orchestra.run.smithi076.stdout:2512 write 0xfb4396 thru 0xfba768 (0x63d3 bytes) 2024-09-14T22:53:58.979 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.979 INFO:teuthology.orchestra.run.smithi076.stdout:2514 writesame 0x34d6c95 thru 0x34e1db8 data_size 0x6(0xb124 bytes) 2024-09-14T22:53:58.984 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:58.984 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:58.985 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:58.985 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:58.985 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:58.986 INFO:teuthology.orchestra.run.smithi076.stdout:2515 read 0x3325c73 thru 0x3335508 (0xf896 bytes) 2024-09-14T22:53:58.987 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.987 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:58.987 INFO:teuthology.orchestra.run.smithi076.stdout:2519 read 0x8dd55e0 thru 0x8de0e0b (0xb82c bytes) 2024-09-14T22:53:58.987 INFO:teuthology.orchestra.run.smithi076.stdout:2520 read 0xd29e376 thru 0xd2ad33e (0xefc9 bytes) 2024-09-14T22:53:58.997 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:53:58.998 INFO:teuthology.orchestra.run.smithi076.stdout:2521 trunc from 0xd6b265c to 0x5a5f7ec 2024-09-14T22:53:59.007 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:59.016 INFO:teuthology.orchestra.run.smithi076.stdout:2524 write 0x8e4b213 thru 0x8e5928a (0xe078 bytes) 2024-09-14T22:53:59.017 INFO:teuthology.orchestra.run.smithi076.stdout:2525 read 0x7b85ed7 thru 0x7b866fc (0x826 bytes) 2024-09-14T22:53:59.018 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:59.018 INFO:teuthology.orchestra.run.smithi076.stdout:2527 read 0x12d7e2b thru 0x12e2dac (0xaf82 bytes) 2024-09-14T22:53:59.018 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:59.019 INFO:teuthology.orchestra.run.smithi076.stdout:2529 writesame 0x347fc0b thru 0x348c1d2 data_size 0x4(0xc5c8 bytes) 2024-09-14T22:53:59.026 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:53:59.024+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:53:59.033 INFO:teuthology.orchestra.run.smithi076.stdout:2530 writesame 0x9e8cffb thru 0x9e9cc4d data_size 0x1(0xfc53 bytes) 2024-09-14T22:53:59.035 INFO:teuthology.orchestra.run.smithi076.stdout:2531 read 0x956b0ea thru 0x9570df9 (0x5d10 bytes) 2024-09-14T22:53:59.035 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:59.035 INFO:teuthology.orchestra.run.smithi076.stdout:2533 write 0x3ef448c thru 0x3f02d96 (0xe90b bytes) 2024-09-14T22:53:59.043 INFO:teuthology.orchestra.run.smithi076.stdout:2534 writesame 0xaec2f4f thru 0xaed2422 data_size 0x12(0xf4d4 bytes) 2024-09-14T22:53:59.044 INFO:teuthology.orchestra.run.smithi076.stdout:2535 punch from 0x4d5a9ff to 0x4d67159, (0xc75a bytes) 2024-09-14T22:53:59.054 INFO:teuthology.orchestra.run.smithi076.stdout:2536 writesame 0xedb8bc7 thru 0xedbee88 data_size 0x1(0x62c2 bytes) 2024-09-14T22:53:59.066 INFO:teuthology.orchestra.run.smithi076.stdout:2537 trunc from 0xedbee89 to 0x58050ea 2024-09-14T22:53:59.073 INFO:teuthology.orchestra.run.smithi076.stdout:2538 write 0x4667ae thru 0x470876 (0xa0c9 bytes) 2024-09-14T22:53:59.075 INFO:teuthology.orchestra.run.smithi076.stdout:2539 write 0x29feb9c thru 0x2a0554a (0x69af bytes) 2024-09-14T22:53:59.077 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:59.077 INFO:teuthology.orchestra.run.smithi076.stdout:2541 write 0x3c530cd thru 0x3c6106d (0xdfa1 bytes) 2024-09-14T22:53:59.079 INFO:teuthology.orchestra.run.smithi076.stdout:2542 read 0x36dea62 thru 0x36e759a (0x8b39 bytes) 2024-09-14T22:53:59.081 INFO:teuthology.orchestra.run.smithi076.stdout:2543 trunc from 0x58050ea to 0x3f16789 2024-09-14T22:53:59.090 INFO:teuthology.orchestra.run.smithi076.stdout:2544 read 0x2b37d55 thru 0x2b3834a (0x5f6 bytes) 2024-09-14T22:53:59.091 INFO:teuthology.orchestra.run.smithi076.stdout:2545 trunc from 0x3f16789 to 0x3814aff 2024-09-14T22:53:59.115 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:59.115 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:59.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8992 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'23, mlcod=33'21)", 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.206084380999997, 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0035453559999999999, 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906376+0000", 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.42e-06 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906968+0000", 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00059252399999999998 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906990+0000", 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1498999999999999e-05 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906999+0000", 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2299999999999997e-06 2024-09-14T22:53:59.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907058+0000", 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9505e-05 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907082+0000", 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3298999999999998e-05 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909874+0000", 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027926689999999998 2024-09-14T22:53:59.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909919+0000", 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5210000000000003e-05 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8998 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'25, mlcod=33'21)", 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.204416447, 2024-09-14T22:53:59.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003268272, 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908043+0000", 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908054+0000", 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1654999999999999e-05 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908066+0000", 2024-09-14T22:53:59.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1321000000000001e-05 2024-09-14T22:53:59.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908077+0000", 2024-09-14T22:53:59.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1781e-05 2024-09-14T22:53:59.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908149+0000", 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1402000000000005e-05 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908170+0000", 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0746000000000001e-05 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911259+0000", 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030889839999999999 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911310+0000", 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1684999999999997e-05 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15153 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'58, mlcod=34'56)", 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.402242029, 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043627650000000002, 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.125 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710218+0000", 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.291e-06 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710957+0000", 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00073905599999999998 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710977+0000", 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0506999999999999e-05 2024-09-14T22:53:59.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710986+0000", 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0860000000000008e-06 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711048+0000", 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1130999999999995e-05 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711072+0000", 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4167000000000001e-05 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714533+0000", 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034611070000000002 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714579+0000", 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6419999999999999e-05 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.129 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15154 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'60, mlcod=34'56)", 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.400891274000003, 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00342618, 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711568+0000", 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0900000000000002e-07 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711574+0000", 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5119999999999997e-06 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711583+0000", 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7579999999999998e-06 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711589+0000", 2024-09-14T22:53:59.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9619999999999998e-06 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711692+0000", 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000103237 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711719+0000", 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7124999999999999e-05 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714953+0000", 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032341240000000001 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714993+0000", 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9953000000000002e-05 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.134 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77339 3.9 e40/24 3:939e5cf2:::rbd_data.113564f98e71.0000000000000044:head v 40'60, mlcod=40'58)", 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.080275507000003, 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0019342230000000001, 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:59.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032183+0000", 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032184+0000", 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2780000000000001e-06 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032626+0000", 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000441369 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032644+0000", 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8029999999999998e-05 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032652+0000", 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7760000000000003e-06 2024-09-14T22:53:59.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032720+0000", 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.032743+0000", 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2823000000000002e-05 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034080+0000", 2024-09-14T22:53:59.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001337705 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034117+0000", 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6854000000000003e-05 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77340 3.9 e40/24 3:939e5cf2:::rbd_data.113564f98e71.0000000000000044:head v 40'62, mlcod=40'58)", 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:59.139 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.079386464999999, 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00147386, 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033072+0000", 2024-09-14T22:53:59.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033073+0000", 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9999999999999998e-07 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033078+0000", 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5430000000000001e-06 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033087+0000", 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.5420000000000005e-06 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033093+0000", 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9479999999999996e-06 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033185+0000", 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2298000000000003e-05 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.033218+0000", 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3389999999999997e-05 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034501+0000", 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012830160000000001 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.034546+0000", 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4623000000000003e-05 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.143 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 31.903937034999998, 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.145 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:53:59.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:78175 3.4 e41/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 41'105, mlcod=40'102)", 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 31.146658080999998, 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0049595489999999997, 2024-09-14T22:53:59.148 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965801+0000", 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.35e-07 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965834+0000", 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3683999999999998e-05 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965846+0000", 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1620999999999999e-05 2024-09-14T22:53:59.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965855+0000", 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1130000000000007e-06 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965939+0000", 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4029000000000005e-05 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965963+0000", 2024-09-14T22:53:59.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3529000000000001e-05 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970705+0000", 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.004742853 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970760+0000", 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4484999999999998e-05 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:116229 3.1e e48/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 48'99, mlcod=48'97)", 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 22.80836618, 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034125309999999999, 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:59.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304095+0000", 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3029999999999998e-06 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304126+0000", 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1374999999999997e-05 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304140+0000", 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4518000000000001e-05 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304150+0000", 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:53:59.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304213+0000", 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3366000000000003e-05 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304237+0000", 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4379000000000002e-05 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307471+0000", 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032338829999999999 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307505+0000", 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3552e-05 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:53:59.157 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.780275261, 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:131796 3.1e e53/44 3:79e10459:::rbd_data.1135a5fb387.0000000000000058:head v 53'145, mlcod=52'143)", 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 17.551011585000001, 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012088559999999999, 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.162 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561447+0000", 2024-09-14T22:53:59.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8e-07 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561522+0000", 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4567999999999995e-05 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561534+0000", 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2517999999999999e-05 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561546+0000", 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1440000000000001e-05 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561630+0000", 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4615000000000002e-05 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.561659+0000", 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8969e-05 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.562606+0000", 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00094671999999999996 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:41.562656+0000", 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9845999999999997e-05 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.166 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:133036 3.4 e54/24 3:22b239af:::rbd_data.1135a4365f9c.0000000000000001:head v 54'137, mlcod=53'135)", 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 16.842575312000001, 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002821278, 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7100000000000001e-07 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270380+0000", 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00049641500000000005 2024-09-14T22:53:59.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270393+0000", 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3088000000000001e-05 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270402+0000", 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6199999999999994e-06 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270470+0000", 2024-09-14T22:53:59.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7459999999999994e-05 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270498+0000", 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8214000000000001e-05 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272657+0000", 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021591499999999999 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272704+0000", 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7160000000000002e-05 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:135238 3.4 e56/24 3:2474bdb6:::rbd_object_map.113530dd5d95:head v 56'311, mlcod=56'309)", 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.034818+0000", 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.077640303000001, 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0015748439999999999, 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.171 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034818+0000", 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034818+0000", 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034819+0000", 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8299999999999995e-07 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034829+0000", 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0139999999999999e-05 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034844+0000", 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4620999999999999e-05 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034852+0000", 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.909e-06 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034904+0000", 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2080000000000003e-05 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.034924+0000", 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0227000000000001e-05 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.036344+0000", 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0014203219999999999 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.036393+0000", 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8662000000000003e-05 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.8485077899999993, 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:53:59.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:53:59.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:138018 3.9 e61/24 3:94720cbd:::rbd_data.1135d9b82239.000000000000003f:head v 61'108, mlcod=60'106)", 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:50.257911+0000", 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.8545472079999996, 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001337092, 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.180 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257911+0000", 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257911+0000", 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257912+0000", 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2999999999999999e-07 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257920+0000", 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8169999999999999e-06 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257931+0000", 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.101e-05 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257937+0000", 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6470000000000003e-06 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.257989+0000", 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1929999999999999e-05 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.258011+0000", 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2218e-05 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.259210+0000", 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001198787 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.259248+0000", 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8753e-05 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:138416 3.5 e61/47 3:a3cf4870:::rbd_data.1135d9b82239.0000000000000013:head v 61'157, mlcod=60'155)", 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:50.401811+0000", 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.7106471439999993, 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0022471959999999999, 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401811+0000", 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401811+0000", 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401813+0000", 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2750000000000001e-06 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401824+0000", 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1255e-05 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401837+0000", 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2988e-05 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401843+0000", 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1580000000000002e-06 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401893+0000", 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9963000000000003e-05 2024-09-14T22:53:59.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.401914+0000", 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1007000000000001e-05 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.404020+0000", 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002105787 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.404059+0000", 2024-09-14T22:53:59.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8763000000000001e-05 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:140152 3.5 e61/47 3:a3cf4870:::rbd_data.1135d9b82239.0000000000000013:head v 61'163, mlcod=61'161)", 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:50.810788+0000", 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.3016708339999994, 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021170479999999998, 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.189 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810788+0000", 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810788+0000", 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810789+0000", 2024-09-14T22:53:59.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1030000000000001e-06 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810834+0000", 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5629e-05 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810849+0000", 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4705999999999999e-05 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810856+0000", 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5230000000000001e-06 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810906+0000", 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9997000000000003e-05 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.810922+0000", 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6274999999999999e-05 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.812871+0000", 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0019489640000000001 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:50.812905+0000", 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3850999999999998e-05 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.193 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:141657 3.1e e65/44 3:7b172f22:::rbd_data.113580c09eae.000000000000000c:head v 65'217, mlcod=64'214)", 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:54.024381+0000", 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.0880777989999997, 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00127625, 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024381+0000", 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024381+0000", 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024381+0000", 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4400000000000001e-07 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024386+0000", 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2499999999999997e-06 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024397+0000", 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0564000000000001e-05 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024410+0000", 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3563999999999999e-05 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024467+0000", 2024-09-14T22:53:59.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7074000000000003e-05 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.024488+0000", 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0203999999999998e-05 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.025632+0000", 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011444999999999999 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.025657+0000", 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4749999999999999e-05 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:141677 3.1e e65/44 3:7e4db642:::rbd_data.113580c09eae.0000000000000007:head v 65'219, mlcod=64'215)", 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:54.037906+0000", 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.0745527189999997, 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001637555, 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.198 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037906+0000", 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037906+0000", 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037907+0000", 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.186e-06 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037917+0000", 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0156999999999999e-05 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037933+0000", 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6322999999999999e-05 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.037942+0000", 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0280000000000006e-06 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.038000+0000", 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7816999999999999e-05 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.038023+0000", 2024-09-14T22:53:59.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2687000000000001e-05 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.039514+0000", 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001490812 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.039543+0000", 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9544999999999999e-05 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:142014 3.1e e65/44 3:7ae60eba:::rbd_data.1135a7a7c51e.0000000000000000:head v 65'220, mlcod=65'217)", 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.9225045039999999, 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0042481029999999996, 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189955+0000", 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0979999999999999e-06 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191197+0000", 2024-09-14T22:53:59.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012418710000000001 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191217+0000", 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.978e-05 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191226+0000", 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6500000000000008e-06 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191288+0000", 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1227000000000002e-05 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191312+0000", 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4088999999999999e-05 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.194153+0000", 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002841437 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.194202+0000", 2024-09-14T22:53:59.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8950999999999999e-05 2024-09-14T22:53:59.207 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.207 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.207 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.207 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.207 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.207 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:59.208 INFO:teuthology.orchestra.run.smithi076.stdout:2549 write 0xe550384 thru 0xe555760 (0x53dd bytes) 2024-09-14T22:53:59.208 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:53:59.208 INFO:teuthology.orchestra.run.smithi076.stdout:2553 read 0x2445baa thru 0x244685f (0xcb6 bytes) 2024-09-14T22:53:59.209 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:53:59.210 INFO:teuthology.orchestra.run.smithi076.stdout:2554 trunc from 0xe555761 to 0xbb550aa 2024-09-14T22:53:59.210 INFO:teuthology.orchestra.run.smithi076.stdout:2555 trunc from 0xbb550aa to 0x3879190 2024-09-14T22:53:59.210 INFO:teuthology.orchestra.run.smithi076.stdout:2557 clone 25 order 23 su 32768 sc 3 2024-09-14T22:53:59.210 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:59.207+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:59.309 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:59.308+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:59.384 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:59.384 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:59.384 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:53:59.384 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:59.384 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:53:59.410 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:53:59.409+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:53:59.511 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:59.510+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:59.528 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:59.528 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:53:59.528 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:53:59.528 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:53:59.528 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:59.536 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:53:59.620 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:53:59.619+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:53:59.681 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:53:59.681 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:53:59.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:53:59.681 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:53:59.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.774085274999997, 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.682 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:53:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:53:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:53:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:53:59.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:53:59.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.774064738999996, 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.687 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:53:59.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:53:59.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.691 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.774034739999998, 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:53:59.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.774003587000003, 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:53:59.696 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:53:59.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:53:59.701 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.701 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.701 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.701 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.701 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:59.701 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773980930999997, 2024-09-14T22:53:59.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:53:59.702 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.702 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.702 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.702 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:53:59.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.703 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:59.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:59.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:53:59.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:53:59.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773960430000002, 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.708 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.712 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773940668999998, 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:53:59.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773919884000001, 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.717 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:53:59.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773887471999998, 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:53:59.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773866073000001, 2024-09-14T22:53:59.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:53:59.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:53:59.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773804673999997, 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.731 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:53:59.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.735 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773764761000002, 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.736 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:53:59.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:53:59.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773735004999999, 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.740 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:53:59.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:59.744 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773704770000002, 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:53:59.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.748 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773680030999998, 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:53:59.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.753 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773656600999999, 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:59.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:53:59.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.757 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773560865, 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:53:59.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:53:59.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773531943000002, 2024-09-14T22:53:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:53:59.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:53:59.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.773508618000001, 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.767 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:53:59.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:53:59.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.640810875, 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:53:59.771 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:53:59.775 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.776 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.776 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.776 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:53:59.776 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:53:59.776 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:53:59.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:53:59.778 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:53:59.776+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:53:59.879 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:53:59.877+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:53:59.887 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:59.887 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:59.888 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:53:59.888 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:53:59.896 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:53:59.980 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:53:59.978+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:53:59.999 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:53:59.999 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:53:59.999 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:00.000 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:00.000 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:00.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:54:00.013 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:00.056 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone24 from 0x3879190 (overlap 0x3814aff) to 0x21a6032 2024-09-14T22:54:00.081 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:00.079+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116113 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5611520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.068660111, 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053031104000000003, 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116113 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053986+0000", 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010658 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053989+0000", 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3579999999999999e-06 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053998+0000", 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8319999999999995e-06 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054004+0000", 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3579999999999999e-06 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106732+0000", 2024-09-14T22:54:00.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052727505000000001 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106816+0000", 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4482999999999997e-05 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106910+0000", 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3987999999999995e-05 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.068524412999999, 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.130 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:54:00.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.068329321, 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.134 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:54:00.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.068176530999999, 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:54:00.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.067971508999999, 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.143 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:54:00.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.067700843000001, 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:54:00.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:54:00.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.067438682999999, 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.152 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.067279831, 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:54:00.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:54:00.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.067093034999999, 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:54:00.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:00.165 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.066890789999999, 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:00.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.066682432, 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.170 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:54:00.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:54:00.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.066368714999999, 2024-09-14T22:54:00.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:54:00.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.066162952999999, 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:54:00.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.065994076999999, 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:54:00.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:54:00.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.065723383000002, 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.188 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.192 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.065574108, 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:54:00.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.065332865999999, 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:00.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116130 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5566464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.201 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.065111831999999, 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053025477000000001, 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116130 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057427+0000", 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057631+0000", 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020390499999999999 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057636+0000", 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4299999999999999e-06 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057647+0000", 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1516e-05 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057653+0000", 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3620000000000002e-06 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110336+0000", 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052682584999999997 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110379+0000", 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2901000000000001e-05 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110453+0000", 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3777999999999997e-05 2024-09-14T22:54:00.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116131 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5685248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.064874319000001, 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053011547999999999, 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.206 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116131 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057665+0000", 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057795+0000", 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013012700000000001 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057805+0000", 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0482999999999999e-05 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057817+0000", 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1499e-05 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057827+0000", 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0091999999999999e-05 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110493+0000", 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052666339999999999 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110578+0000", 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4876000000000005e-05 2024-09-14T22:54:00.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110676+0000", 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8130999999999999e-05 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.756642834000001, 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:54:00.210 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:54:00.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:54:00.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:00.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:00.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:00.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:54:00.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:54:00.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.216 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:00.216 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:00.216 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:00.217 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T22:54:00.221 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:00.221 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:00.216+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:00.318 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:00.317+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:00.325 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:00.326 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:00.334 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:00.372 INFO:teuthology.orchestra.run.smithi076.stdout:2558 write 0xe0116e7 thru 0xe01d7ef (0xc109 bytes) 2024-09-14T22:54:00.377 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.377 INFO:teuthology.orchestra.run.smithi076.stdout:2562 punch from 0xd5cfd55 to 0xd5de967, (0xec12 bytes) 2024-09-14T22:54:00.377 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.377 INFO:teuthology.orchestra.run.smithi076.stdout:2564 writesame 0xd7b9f6f thru 0xd7be24c data_size 0x3(0x42de bytes) 2024-09-14T22:54:00.382 INFO:teuthology.orchestra.run.smithi076.stdout:2567 writesame 0x235f028 thru 0x23685d9 data_size 0x9(0x95b2 bytes) 2024-09-14T22:54:00.411 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:00.419 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:00.417+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:00.477 INFO:teuthology.orchestra.run.smithi076.stdout:2569 trunc from 0xe01d7f0 to 0x44e3242 2024-09-14T22:54:00.498 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:00.498 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:00.498 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:00.498 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:00.499 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:00.509 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:00.517 INFO:teuthology.orchestra.run.smithi076.stdout:2570 read 0x3616af1 thru 0x361d392 (0x68a2 bytes) 2024-09-14T22:54:00.518 INFO:teuthology.orchestra.run.smithi076.stdout:2572 read 0x3f25256 thru 0x3f33fc7 (0xed72 bytes) 2024-09-14T22:54:00.520 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:00.519+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:00.528 INFO:teuthology.orchestra.run.smithi076.stdout:2573 write 0x86306b9 thru 0x863f855 (0xf19d bytes) 2024-09-14T22:54:00.532 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.543 INFO:teuthology.orchestra.run.smithi076.stdout:2575 write 0xd9634d8 thru 0xd969c40 (0x6769 bytes) 2024-09-14T22:54:00.550 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.550 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.550 INFO:teuthology.orchestra.run.smithi076.stdout:2582 write 0x836dcd4 thru 0x83755ef (0x791c bytes) 2024-09-14T22:54:00.552 INFO:teuthology.orchestra.run.smithi076.stdout:2583 writesame 0x120e3a7 thru 0x1218571 data_size 0x7(0xa1cb bytes) 2024-09-14T22:54:00.566 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:00.567 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:00.594 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:00.595 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:00.614 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:00.615 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:00.621 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:00.620+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:00.631 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:00.632 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:00.644 INFO:teuthology.orchestra.run.smithi076.stdout:2587 read 0xb94fa20 thru 0xb9560b1 (0x6692 bytes) 2024-09-14T22:54:00.644 INFO:teuthology.orchestra.run.smithi076.stdout:2588 write 0xa22c79d thru 0xa22da07 (0x126b bytes) 2024-09-14T22:54:00.647 INFO:teuthology.orchestra.run.smithi076.stdout:2589 write 0x2df5562 thru 0x2dfd269 (0x7d08 bytes) 2024-09-14T22:54:00.648 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:00.648 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:00.649 INFO:teuthology.orchestra.run.smithi076.stdout:2590 writesame 0x8b42659 thru 0x8b44ae6 data_size 0x1(0x248e bytes) 2024-09-14T22:54:00.656 INFO:teuthology.orchestra.run.smithi076.stdout:2591 writesame 0x5661efd thru 0x566e33c data_size 0x10(0xc440 bytes) 2024-09-14T22:54:00.661 INFO:teuthology.orchestra.run.smithi076.stdout:2592 write 0x894aec8 thru 0x8950874 (0x59ad bytes) 2024-09-14T22:54:00.663 INFO:teuthology.orchestra.run.smithi076.stdout:2594 write 0xc54ef49 thru 0xc55a72b (0xb7e3 bytes) 2024-09-14T22:54:00.667 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:00.667 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:00.669 INFO:teuthology.orchestra.run.smithi076.stdout:2596 write 0xcceaeb5 thru 0xccec183 (0x12cf bytes) 2024-09-14T22:54:00.670 INFO:teuthology.orchestra.run.smithi076.stdout:2597 read 0x8d7268c thru 0x8d74df9 (0x276e bytes) 2024-09-14T22:54:00.671 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.671 INFO:teuthology.orchestra.run.smithi076.stdout:2600 read 0x940456 thru 0x948bb1 (0x875c bytes) 2024-09-14T22:54:00.671 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.672 INFO:teuthology.orchestra.run.smithi076.stdout:2602 trunc from 0xd969c41 to 0xce98340 2024-09-14T22:54:00.673 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:00.673 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:00.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:00.685 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:00.685 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:00.692 INFO:teuthology.orchestra.run.smithi076.stdout:2604 write 0xd0c2d33 thru 0xd0c9804 (0x6ad2 bytes) 2024-09-14T22:54:00.694 INFO:teuthology.orchestra.run.smithi076.stdout:2605 writesame 0x83d5521 thru 0x83dec3c data_size 0x4(0x971c bytes) 2024-09-14T22:54:00.702 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.702 INFO:teuthology.orchestra.run.smithi076.stdout:2607 punch from 0x8287515 to 0x828bc1a, (0x4705 bytes) 2024-09-14T22:54:00.703 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.703 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.704 INFO:teuthology.orchestra.run.smithi076.stdout:2610 write 0xb9a3139 thru 0xb9af9bf (0xc887 bytes) 2024-09-14T22:54:00.707 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:00.708 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:00.708 INFO:teuthology.orchestra.run.smithi076.stdout:2612 read 0x61e65e0 thru 0x61ec354 (0x5d75 bytes) 2024-09-14T22:54:00.708 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.709 INFO:teuthology.orchestra.run.smithi076.stdout:2615 read 0x95d77d2 thru 0x95d8544 (0xd73 bytes) 2024-09-14T22:54:00.709 INFO:teuthology.orchestra.run.smithi076.stdout:2616 writesame 0x476f0f5 thru 0x47781d0 data_size 0x49(0x90dc bytes) 2024-09-14T22:54:00.711 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:00.711 INFO:teuthology.orchestra.run.smithi076.stdout:2619 writesame 0x2394785 thru 0x23a05fd data_size 0x1(0xbe79 bytes) 2024-09-14T22:54:00.727 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:00.726+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:00.745 INFO:teuthology.orchestra.run.smithi076.stdout:2621 clone 26 order 25 su 16777216 sc 10 2024-09-14T22:54:00.789 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:00.789 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:00.797 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:00.839 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:00.837+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:00.924 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:00.922+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:00.941 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:00.941 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:00.950 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:01.006 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:54:01.063 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:01.062+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:01.079 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone25 from 0xd0c9805 (overlap 0x3879190) to 0x0 2024-09-14T22:54:01.091 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:01.091 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:54:01.091 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T22:54:01.092 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:54:01.109 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:01.109 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:01.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:01.157 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T22:54:01.161 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:01.160+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:01.216 INFO:teuthology.orchestra.run.smithi076.stdout:2622 writesame 0xe3ae181 thru 0xe3b6728 data_size 0x38(0x85a8 bytes) 2024-09-14T22:54:01.219 INFO:teuthology.orchestra.run.smithi076.stdout:2623 write 0xc9a1855 thru 0xc9ab966 (0xa112 bytes) 2024-09-14T22:54:01.226 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:01.225+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:01.230 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:01.240 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:01.304 INFO:teuthology.orchestra.run.smithi076.stdout:2625 write 0x5d3654 thru 0x5e0dde (0xd78b bytes) 2024-09-14T22:54:01.327 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:01.326+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:01.384 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:01.385 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:01.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:01.396 INFO:teuthology.orchestra.run.smithi076.stdout:2626 writesame 0x6e372bf thru 0x6e3b475 data_size 0x1(0x41b7 bytes) 2024-09-14T22:54:01.406 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:01.406 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:01.406 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 71, 2024-09-14T22:54:01.406 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:54:01.406 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:54:01.406 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:54:00.999160+0000", 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5804032, 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "nodeep-scrub", 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:01.407 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:54:01.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:01.409 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:01.410 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:01.411 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:01.412 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:01.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:01.414 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:54:01.415 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:54:01.416 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:01.417 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:01.418 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:54:01.419 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:01.420 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:01.421 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:54:01.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 3, 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 3, 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.75, 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2.25, 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2.25, 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:01.424 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:54:01.425 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "71", 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:01.426 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 28, 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 71, 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:01.427 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:01.428 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:01.429 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 1.75, 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 1.75, 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.75, 2024-09-14T22:54:01.430 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.75, 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:01.431 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.5721282958984375, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.433 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.434 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:01.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:54:01.434 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:01.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.435 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.435 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:54:01.435 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:01.435 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.435 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.436 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:01.436 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.436 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.436 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:54:01.436 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:01.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.437 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.437 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:54:01.437 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:01.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.437 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.438 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:54:01.439 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:01.439 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.439 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.439 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:54:01.439 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:54:01.439 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:54:01.439 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:54:01.440 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:01.440 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:01.440 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:01.440 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.440 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:54:01.441 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:54:01.441 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:01.441 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:01.441 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:01.441 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.0684051513671875, 2024-09-14T22:54:01.441 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:01.441 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:01.441 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:01.442 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 58, 2024-09-14T22:54:01.442 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:01.442 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:01.442 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:01.442 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.442 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.442 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:54:01.443 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:01.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.443 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.443 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:54:01.443 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:01.443 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.444 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.444 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:01.444 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.444 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.445 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.446 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.447 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.448 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:54:01.448 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:01.448 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.448 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.448 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:54:01.448 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:54:01.448 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:54:01.449 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:54:01.449 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:01.449 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:01.449 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:01.449 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.450 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:54:01.450 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:54:01.450 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:01.450 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:01.450 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:01.450 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:01.450 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:01.451 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:01.451 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:01.451 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 62, 2024-09-14T22:54:01.451 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:01.451 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:01.451 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:01.451 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.452 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.453 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:01.453 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.453 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.453 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:54:01.454 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:01.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.454 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.454 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:54:01.454 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:01.454 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.454 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.455 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:01.455 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.455 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.455 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:54:01.455 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:01.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.456 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.456 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:54:01.456 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:01.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.456 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.457 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:01.457 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.457 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.457 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:54:01.457 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:01.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:54:01.458 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:54:01.459 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:01.459 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:01.459 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:01.459 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.459 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:54:01.459 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0.5237274169921875, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 62, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:01.460 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:54:01.461 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.462 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:54:01.463 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:01.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.466 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.466 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:54:01.466 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:54:01.466 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:54:01.466 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:54:01.466 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:01.466 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:01.467 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:01.467 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.467 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:54:01.467 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:54:01.467 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:01.468 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:01.468 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:01.468 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:01.468 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:01.468 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:01.468 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:01.468 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:54:01.468 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:01.469 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:01.469 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:01.469 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.469 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.469 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:54:01.470 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:01.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.470 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.470 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:54:01.470 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:01.470 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.471 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.471 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:01.471 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.471 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.471 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:54:01.471 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:01.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.472 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.472 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:54:01.472 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:01.472 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.472 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.473 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:01.473 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.473 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.473 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:54:01.473 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.475 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:54:01.476 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:54:01.477 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:01.477 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:01.477 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:01.477 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:01.477 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:01.477 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:01.477 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:01.477 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 50, 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.478 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:01.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.480 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.481 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.482 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:01.484 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.485 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:54:01.487 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.489 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:54:01.490 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:54:01.490 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:54:01.490 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:54:01.490 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:01.490 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:01.490 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:01.490 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:54:01.491 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:54:01.491 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:01.491 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:01.491 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:01.492 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:01.492 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:01.492 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:01.492 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:01.492 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 50, 2024-09-14T22:54:01.492 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:01.492 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:01.493 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:01.493 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.493 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.493 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:54:01.495 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:01.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.496 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.496 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:01.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:54:01.497 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:01.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.497 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.497 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:54:01.497 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:01.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.498 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.498 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:01.498 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:01.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.498 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:01.499 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:54:01.499 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:01.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.499 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:01.499 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:54:01.499 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:01.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.500 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.500 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:54:01.500 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:54:01.500 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:54:01.500 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:01.501 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:01.502 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:01.503 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:01.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:54:01.506 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:01.507 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "2.6", 2024-09-14T22:54:01.508 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: 5, 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.8", 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: 3, 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [], 2024-09-14T22:54:01.510 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:54:01.511 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [], 2024-09-14T22:54:01.511 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:54:01.511 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:54:01.511 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:54:01.511 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:54:01.511 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:54:01.511 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:54:01.512 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:54:01.512 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:54:01.512 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:54:01.512 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:54:01.512 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:01.512 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [], 2024-09-14T22:54:01.513 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:54:01.513 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:54:01.513 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:54:01.513 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:54:01.513 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:54:01.513 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:54:01.513 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:54:01.513 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:54:01.514 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:54:01.514 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:54:01.514 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:01.514 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:01.516 INFO:teuthology.orchestra.run.smithi076.stdout:2628 read 0xbc6fade thru 0xbc77c59 (0x817c bytes) 2024-09-14T22:54:01.516 INFO:teuthology.orchestra.run.smithi076.stdout:2629 write 0x7c7ebe4 thru 0x7c871b7 (0x85d4 bytes) 2024-09-14T22:54:01.516 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.516 INFO:teuthology.orchestra.run.smithi076.stdout:2632 write 0x56e84fe thru 0x56f54d3 (0xcfd6 bytes) 2024-09-14T22:54:01.518 INFO:tasks.thrashosds.thrasher:j is {'epoch': 71, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:54:00.999160+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5721282958984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0684051513671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 0.5237274169921875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [5, 2]}, {'pgid': '3.0', 'osds': [6, 7]}, {'pgid': '3.8', 'osds': [3, 6]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:54:01.518 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:54:01.519 INFO:teuthology.orchestra.run.smithi076.stdout:2633 write 0xebcec88 thru 0xebd99cb (0xad44 bytes) 2024-09-14T22:54:01.519 INFO:teuthology.orchestra.run.smithi076.stdout:2634 punch from 0x384d3e2 to 0x385ab38, (0xd756 bytes) 2024-09-14T22:54:01.519 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.519 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.519 INFO:teuthology.orchestra.run.smithi076.stdout:2639 write 0xbab5736 thru 0xbac1535 (0xbe00 bytes) 2024-09-14T22:54:01.519 INFO:teuthology.orchestra.run.smithi076.stdout:2640 write 0x180a591 thru 0x180c50b (0x1f7b bytes) 2024-09-14T22:54:01.519 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.520 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:01.514+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:01.529 INFO:teuthology.orchestra.run.smithi076.stdout:2642 trunc from 0xebd99cc to 0x3e6ef5e 2024-09-14T22:54:01.585 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.602 INFO:teuthology.orchestra.run.smithi076.stdout:2644 writesame 0xe192908 thru 0xe19770f data_size 0x2c(0x4e08 bytes) 2024-09-14T22:54:01.603 INFO:teuthology.orchestra.run.smithi076.stdout:2645 read 0x3cad3fc thru 0x3cb5250 (0x7e55 bytes) 2024-09-14T22:54:01.603 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.603 INFO:teuthology.orchestra.run.smithi076.stdout:2648 write 0x79b3a8d thru 0x79c392f (0xfea3 bytes) 2024-09-14T22:54:01.606 INFO:teuthology.orchestra.run.smithi076.stdout:2649 writesame 0x692a673 thru 0x692da0b data_size 0x33(0x3399 bytes) 2024-09-14T22:54:01.609 INFO:teuthology.orchestra.run.smithi076.stdout:2650 read 0x19c64cc thru 0x19c8790 (0x22c5 bytes) 2024-09-14T22:54:01.609 INFO:teuthology.orchestra.run.smithi076.stdout:2651 write 0x6d77574 thru 0x6d7d0ce (0x5b5b bytes) 2024-09-14T22:54:01.611 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.611 INFO:teuthology.orchestra.run.smithi076.stdout:2653 read 0x241c4d0 thru 0x2420e8d (0x49be bytes) 2024-09-14T22:54:01.611 INFO:teuthology.orchestra.run.smithi076.stdout:2655 read 0x28c9f5e thru 0x28cee06 (0x4ea9 bytes) 2024-09-14T22:54:01.612 INFO:teuthology.orchestra.run.smithi076.stdout:2656 punch from 0xdc1896f to 0xdc1e18a, (0x581b bytes) 2024-09-14T22:54:01.616 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:01.615+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:01.618 INFO:teuthology.orchestra.run.smithi076.stdout:2657 trunc from 0xe197710 to 0x902d4fa 2024-09-14T22:54:01.628 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:01.628 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:01.635 INFO:teuthology.orchestra.run.smithi076.stdout:2658 write 0xd04ad6c thru 0xd04ec3d (0x3ed2 bytes) 2024-09-14T22:54:01.636 INFO:teuthology.orchestra.run.smithi076.stdout:2659 read 0x793752e thru 0x7941e3b (0xa90e bytes) 2024-09-14T22:54:01.638 INFO:teuthology.orchestra.run.smithi076.stdout:2660 read 0x9a977ee thru 0x9aa1aae (0xa2c1 bytes) 2024-09-14T22:54:01.638 INFO:teuthology.orchestra.run.smithi076.stdout:2662 read 0x5dadc65 thru 0x5db4a52 (0x6dee bytes) 2024-09-14T22:54:01.638 INFO:teuthology.orchestra.run.smithi076.stdout:2663 punch from 0x8e8ff21 to 0x8e9a975, (0xaa54 bytes) 2024-09-14T22:54:01.638 INFO:teuthology.orchestra.run.smithi076.stdout:2664 read 0x55a0a64 thru 0x55a8698 (0x7c35 bytes) 2024-09-14T22:54:01.638 INFO:teuthology.orchestra.run.smithi076.stdout:2666 punch from 0xba710 to 0xca240, (0xfb30 bytes) 2024-09-14T22:54:01.638 INFO:teuthology.orchestra.run.smithi076.stdout:2669 writesame 0x31d88b thru 0x31f234 data_size 0x1e(0x19aa bytes) 2024-09-14T22:54:01.639 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:01.639 INFO:teuthology.orchestra.run.smithi076.stdout:2670 writesame 0x863e88 thru 0x8718bd data_size 0x3e(0xda36 bytes) 2024-09-14T22:54:01.642 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.654 INFO:teuthology.orchestra.run.smithi076.stdout:2672 trunc from 0xd04ec3e to 0x3f237e5 2024-09-14T22:54:01.664 INFO:teuthology.orchestra.run.smithi076.stdout:2673 write 0x3df6234 thru 0x3dfa645 (0x4412 bytes) 2024-09-14T22:54:01.666 INFO:teuthology.orchestra.run.smithi076.stdout:2674 read 0x692504 thru 0x69b374 (0x8e71 bytes) 2024-09-14T22:54:01.667 INFO:teuthology.orchestra.run.smithi076.stdout:2675 punch from 0x1c9c9fa to 0x1ca5dd8, (0x93de bytes) 2024-09-14T22:54:01.667 INFO:teuthology.orchestra.run.smithi076.stdout:2676 write 0x2a05417 thru 0x2a0b149 (0x5d33 bytes) 2024-09-14T22:54:01.669 INFO:teuthology.orchestra.run.smithi076.stdout:2677 read 0x3c54e9f thru 0x3c55e82 (0xfe4 bytes) 2024-09-14T22:54:01.670 INFO:teuthology.orchestra.run.smithi076.stdout:2678 read 0x27bb1f1 thru 0x27c1a07 (0x6817 bytes) 2024-09-14T22:54:01.670 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.670 INFO:teuthology.orchestra.run.smithi076.stdout:2681 read 0xc5840e thru 0xc6535f (0xcf52 bytes) 2024-09-14T22:54:01.677 INFO:teuthology.orchestra.run.smithi076.stdout:2682 write 0x4ec258e thru 0x4ec6d9c (0x480f bytes) 2024-09-14T22:54:01.690 INFO:teuthology.orchestra.run.smithi076.stdout:2683 write 0x94089e9 thru 0x94113c7 (0x89df bytes) 2024-09-14T22:54:01.693 INFO:teuthology.orchestra.run.smithi076.stdout:2684 write 0x4bfc511 thru 0x4bfe475 (0x1f65 bytes) 2024-09-14T22:54:01.701 INFO:teuthology.orchestra.run.smithi076.stdout:2685 writesame 0xad598f3 thru 0xad5bab0 data_size 0x7(0x21be bytes) 2024-09-14T22:54:01.709 INFO:teuthology.orchestra.run.smithi076.stdout:2686 write 0xc7edc5c thru 0xc7f8338 (0xa6dd bytes) 2024-09-14T22:54:01.711 INFO:teuthology.orchestra.run.smithi076.stdout:2687 write 0x4e1473e thru 0x4e22716 (0xdfd9 bytes) 2024-09-14T22:54:01.713 INFO:teuthology.orchestra.run.smithi076.stdout:2688 read 0x72ff8f7 thru 0x73037e6 (0x3ef0 bytes) 2024-09-14T22:54:01.713 INFO:teuthology.orchestra.run.smithi076.stdout:2689 write 0x3b189df thru 0x3b1997e (0xfa0 bytes) 2024-09-14T22:54:01.715 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.715 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.715 INFO:teuthology.orchestra.run.smithi076.stdout:2693 punch from 0xb935a7a to 0xb93b05b, (0x55e1 bytes) 2024-09-14T22:54:01.716 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:01.715+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:01.719 INFO:teuthology.orchestra.run.smithi076.stdout:2694 trunc from 0xc7f8339 to 0x8faf396 2024-09-14T22:54:01.730 INFO:teuthology.orchestra.run.smithi076.stdout:2695 write 0x3925967 thru 0x39326c2 (0xcd5c bytes) 2024-09-14T22:54:01.732 INFO:teuthology.orchestra.run.smithi076.stdout:2696 punch from 0x68c8701 to 0x68cd55c, (0x4e5b bytes) 2024-09-14T22:54:01.732 INFO:teuthology.orchestra.run.smithi076.stdout:2697 punch from 0x3f8466c to 0x3f8c129, (0x7abd bytes) 2024-09-14T22:54:01.732 INFO:teuthology.orchestra.run.smithi076.stdout:2698 punch from 0x3619202 to 0x361e5b0, (0x53ae bytes) 2024-09-14T22:54:01.732 INFO:teuthology.orchestra.run.smithi076.stdout:2700 writesame 0x60e9688 thru 0x60f82e8 data_size 0x17(0xec61 bytes) 2024-09-14T22:54:01.736 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.736 INFO:teuthology.orchestra.run.smithi076.stdout:2702 punch from 0xdb5ad4 to 0xdc07df, (0xad0b bytes) 2024-09-14T22:54:01.736 INFO:teuthology.orchestra.run.smithi076.stdout:2703 writesame 0x2fe79e2 thru 0x2fe8c0c data_size 0x1(0x122b bytes) 2024-09-14T22:54:01.740 INFO:teuthology.orchestra.run.smithi076.stdout:2704 punch from 0xe1c9d5 to 0xe26755, (0x9d80 bytes) 2024-09-14T22:54:01.741 INFO:teuthology.orchestra.run.smithi076.stdout:2705 trunc from 0x8faf396 to 0x7e80ff2 2024-09-14T22:54:01.747 INFO:teuthology.orchestra.run.smithi076.stdout:2706 read 0x57e57f0 thru 0x57f457b (0xed8c bytes) 2024-09-14T22:54:01.747 INFO:teuthology.orchestra.run.smithi076.stdout:2707 write 0x3232035 thru 0x32336af (0x167b bytes) 2024-09-14T22:54:01.755 INFO:teuthology.orchestra.run.smithi076.stdout:2710 trunc from 0x7e80ff2 to 0x2fe0560 2024-09-14T22:54:01.757 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:01.758 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:01.768 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:01.774 INFO:teuthology.orchestra.run.smithi076.stdout:2711 writesame 0x4f5caba thru 0x4f65dce data_size 0x4d(0x9315 bytes) 2024-09-14T22:54:01.775 INFO:teuthology.orchestra.run.smithi076.stdout:2712 writesame 0x2c76323 thru 0x2c7b33b data_size 0x5(0x5019 bytes) 2024-09-14T22:54:01.779 INFO:teuthology.orchestra.run.smithi076.stdout:2713 read 0x481764c thru 0x48216b6 (0xa06b bytes) 2024-09-14T22:54:01.779 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.779 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.785 INFO:teuthology.orchestra.run.smithi076.stdout:2716 write 0x65f2558 thru 0x65ff1e5 (0xcc8e bytes) 2024-09-14T22:54:01.788 INFO:teuthology.orchestra.run.smithi076.stdout:2717 punch from 0x4299a18 to 0x429ae7a, (0x1462 bytes) 2024-09-14T22:54:01.788 INFO:teuthology.orchestra.run.smithi076.stdout:2718 read 0x3366039 thru 0x3371222 (0xb1ea bytes) 2024-09-14T22:54:01.789 INFO:teuthology.orchestra.run.smithi076.stdout:2719 read 0x37cae41 thru 0x37d3ce9 (0x8ea9 bytes) 2024-09-14T22:54:01.793 INFO:teuthology.orchestra.run.smithi076.stdout:2720 trunc from 0x65ff1e6 to 0x2a0890e 2024-09-14T22:54:01.804 INFO:teuthology.orchestra.run.smithi076.stdout:2721 trunc from 0x2a0890e to 0x41cc2c3 2024-09-14T22:54:01.817 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:01.815+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:01.827 INFO:teuthology.orchestra.run.smithi076.stdout:2722 write 0xe6474df thru 0xe64fa9f (0x85c1 bytes) 2024-09-14T22:54:01.829 INFO:teuthology.orchestra.run.smithi076.stdout:2723 writesame 0x696e89b thru 0x69703a5 data_size 0x17(0x1b0b bytes) 2024-09-14T22:54:01.845 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:01.846 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:54:01.847 INFO:teuthology.orchestra.run.smithi076.stdout:2725 trunc from 0xe64faa0 to 0x32a0a6f 2024-09-14T22:54:01.856 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.856 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.856 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.856 INFO:teuthology.orchestra.run.smithi076.stdout:2729 write 0x1841f7c thru 0x184df87 (0xc00c bytes) 2024-09-14T22:54:01.858 INFO:teuthology.orchestra.run.smithi076.stdout:2730 read 0x18a1cc8 thru 0x18aecff (0xd038 bytes) 2024-09-14T22:54:01.859 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-14T22:54:01.006712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1010129888,"num_objects":1727,"num_object_clones":242,"num_object_copies":3454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1727,"num_whiteouts":1138,"num_read":169574,"num_read_kb":638944,"num_write":7642,"num_write_kb":360773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":17845622,"num_keys_recovered":18,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7340,"ondisk_log_size":7340,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":780596,"kb_used_data":497728,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754194124,"statfs":{"total":773094113280,"available":772294782976,"internally_reserved":0,"allocated":509673472,"data_stored":596919464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":289394225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3094978,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":4,"num_read":8386,"num_read_kb":25872,"num_write":652,"num_write_kb":13422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.213543"},"pg_stats":[{"pgid":"3.1f","version":"69'401","reported_seq":6794,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.148916+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:53:59.148916+0000","last_peered":"2024-09-14T22:53:59.148916+0000","last_clean":"2024-09-14T22:53:59.148916+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:53:59.148916+0000","last_undegraded":"2024-09-14T22:53:59.148916+0000","last_fullsized":"2024-09-14T22:53:59.148916+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:36:59.506571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":34390815,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":34,"num_read":8120,"num_read_kb":9131,"num_write":501,"num_write_kb":5569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"69'251","reported_seq":7191,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.317486+0000","last_change":"2024-09-14T22:53:32.831259+0000","last_active":"2024-09-14T22:53:58.317486+0000","last_peered":"2024-09-14T22:53:58.317486+0000","last_clean":"2024-09-14T22:53:58.317486+0000","last_became_active":"2024-09-14T22:53:32.751884+0000","last_became_peered":"2024-09-14T22:53:32.751884+0000","last_unstale":"2024-09-14T22:53:58.317486+0000","last_undegraded":"2024-09-14T22:53:58.317486+0000","last_fullsized":"2024-09-14T22:53:58.317486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:24:36.216741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098261599999999996,"stat_sum":{"num_bytes":22714053,"num_objects":60,"num_object_clones":13,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":37,"num_read":7140,"num_read_kb":14066,"num_write":258,"num_write_kb":10199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":301575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"69'165","reported_seq":7325,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.732424+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:53:58.732424+0000","last_peered":"2024-09-14T22:53:58.732424+0000","last_clean":"2024-09-14T22:53:58.732424+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:53:58.732424+0000","last_undegraded":"2024-09-14T22:53:58.732424+0000","last_fullsized":"2024-09-14T22:53:58.732424+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:50.668062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":36469673,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":39,"num_read":7483,"num_read_kb":34183,"num_write":145,"num_write_kb":29279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"69'291","reported_seq":1594,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.734592+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:53:58.734592+0000","last_peered":"2024-09-14T22:53:58.734592+0000","last_clean":"2024-09-14T22:53:58.734592+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:53:58.734592+0000","last_undegraded":"2024-09-14T22:53:58.734592+0000","last_fullsized":"2024-09-14T22:53:58.734592+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:58.961984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":38097071,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":31,"num_read":2120,"num_read_kb":7636,"num_write":362,"num_write_kb":6060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"69'148","reported_seq":1104,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.730210+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:53:58.730210+0000","last_peered":"2024-09-14T22:53:58.730210+0000","last_clean":"2024-09-14T22:53:58.730210+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:53:58.730210+0000","last_undegraded":"2024-09-14T22:53:58.730210+0000","last_fullsized":"2024-09-14T22:53:58.730210+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:55:08.961057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":50122181,"num_objects":43,"num_object_clones":4,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":26,"num_read":964,"num_read_kb":22852,"num_write":158,"num_write_kb":13614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"69'204","reported_seq":10549,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.136708+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:53:59.136708+0000","last_peered":"2024-09-14T22:53:59.136708+0000","last_clean":"2024-09-14T22:53:59.136708+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:53:59.136708+0000","last_undegraded":"2024-09-14T22:53:59.136708+0000","last_fullsized":"2024-09-14T22:53:59.136708+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:10:57.042226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":37649515,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":38,"num_read":10723,"num_read_kb":38822,"num_write":165,"num_write_kb":19843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"69'198","reported_seq":10751,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.732261+0000","last_change":"2024-09-14T22:53:51.943155+0000","last_active":"2024-09-14T22:53:58.732261+0000","last_peered":"2024-09-14T22:53:58.732261+0000","last_clean":"2024-09-14T22:53:58.732261+0000","last_became_active":"2024-09-14T22:53:51.942663+0000","last_became_peered":"2024-09-14T22:53:51.942663+0000","last_unstale":"2024-09-14T22:53:58.732261+0000","last_undegraded":"2024-09-14T22:53:58.732261+0000","last_fullsized":"2024-09-14T22:53:58.732261+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:27:14.623836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":39590624,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":46,"num_read":10932,"num_read_kb":53829,"num_write":176,"num_write_kb":19923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"69'168","reported_seq":734,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.130571+0000","last_change":"2024-09-14T22:53:32.823044+0000","last_active":"2024-09-14T22:53:59.130571+0000","last_peered":"2024-09-14T22:53:59.130571+0000","last_clean":"2024-09-14T22:53:59.130571+0000","last_became_active":"2024-09-14T22:53:32.749192+0000","last_became_peered":"2024-09-14T22:53:32.749192+0000","last_unstale":"2024-09-14T22:53:59.130571+0000","last_undegraded":"2024-09-14T22:53:59.130571+0000","last_fullsized":"2024-09-14T22:53:59.130571+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:32:51.044410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062866099999999998,"stat_sum":{"num_bytes":13645115,"num_objects":50,"num_object_clones":2,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":38,"num_read":445,"num_read_kb":3681,"num_write":140,"num_write_kb":3917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1050624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"69'165","reported_seq":1409,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.131860+0000","last_change":"2024-09-14T22:53:38.882316+0000","last_active":"2024-09-14T22:53:59.131860+0000","last_peered":"2024-09-14T22:53:59.131860+0000","last_clean":"2024-09-14T22:53:59.131860+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:53:59.131860+0000","last_undegraded":"2024-09-14T22:53:59.131860+0000","last_fullsized":"2024-09-14T22:53:59.131860+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:33:32.808826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":21857804,"num_objects":44,"num_object_clones":3,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":28,"num_read":1562,"num_read_kb":6459,"num_write":143,"num_write_kb":8587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"69'216","reported_seq":8385,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.134756+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:53:59.134756+0000","last_peered":"2024-09-14T22:53:59.134756+0000","last_clean":"2024-09-14T22:53:59.134756+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:53:59.134756+0000","last_undegraded":"2024-09-14T22:53:59.134756+0000","last_fullsized":"2024-09-14T22:53:59.134756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:24.467538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":57688558,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":35,"num_read":9228,"num_read_kb":39360,"num_write":180,"num_write_kb":13784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"69'134","reported_seq":8210,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.328154+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:53:58.328154+0000","last_peered":"2024-09-14T22:53:58.328154+0000","last_clean":"2024-09-14T22:53:58.328154+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:53:58.328154+0000","last_undegraded":"2024-09-14T22:53:58.328154+0000","last_fullsized":"2024-09-14T22:53:58.328154+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:13:00.810749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":19186356,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":31,"num_read":8705,"num_read_kb":26754,"num_write":97,"num_write_kb":946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"68'305","reported_seq":7787,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.733535+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:53:58.733535+0000","last_peered":"2024-09-14T22:53:58.733535+0000","last_clean":"2024-09-14T22:53:58.733535+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:53:58.733535+0000","last_undegraded":"2024-09-14T22:53:58.733535+0000","last_fullsized":"2024-09-14T22:53:58.733535+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:08:18.434137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":64433210,"num_objects":60,"num_object_clones":10,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":32,"num_read":8459,"num_read_kb":44147,"num_write":387,"num_write_kb":18039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":113,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.985116+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:53:57.985116+0000","last_peered":"2024-09-14T22:53:57.985116+0000","last_clean":"2024-09-14T22:53:57.985116+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:53:57.985116+0000","last_undegraded":"2024-09-14T22:53:57.985116+0000","last_fullsized":"2024-09-14T22:53:57.985116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"69'440","reported_seq":6711,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.130831+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:53:59.130831+0000","last_peered":"2024-09-14T22:53:59.130831+0000","last_clean":"2024-09-14T22:53:59.130831+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:53:59.130831+0000","last_undegraded":"2024-09-14T22:53:59.130831+0000","last_fullsized":"2024-09-14T22:53:59.130831+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:17:16.190860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":67834599,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":28,"num_read":8416,"num_read_kb":29362,"num_write":549,"num_write_kb":23238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.001813+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:53:58.001813+0000","last_peered":"2024-09-14T22:53:58.001813+0000","last_clean":"2024-09-14T22:53:58.001813+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:53:58.001813+0000","last_undegraded":"2024-09-14T22:53:58.001813+0000","last_fullsized":"2024-09-14T22:53:58.001813+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"69'105","reported_seq":843,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.731279+0000","last_change":"2024-09-14T22:53:08.762000+0000","last_active":"2024-09-14T22:53:58.731279+0000","last_peered":"2024-09-14T22:53:58.731279+0000","last_clean":"2024-09-14T22:53:58.731279+0000","last_became_active":"2024-09-14T22:53:06.761953+0000","last_became_peered":"2024-09-14T22:53:06.761953+0000","last_unstale":"2024-09-14T22:53:58.731279+0000","last_undegraded":"2024-09-14T22:53:58.731279+0000","last_fullsized":"2024-09-14T22:53:58.731279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:12:51.105717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":21185013,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":25,"num_read":674,"num_read_kb":8713,"num_write":78,"num_write_kb":6832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":113,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.991873+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:53:57.991873+0000","last_peered":"2024-09-14T22:53:57.991873+0000","last_clean":"2024-09-14T22:53:57.991873+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:53:57.991873+0000","last_undegraded":"2024-09-14T22:53:57.991873+0000","last_fullsized":"2024-09-14T22:53:57.991873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"61'419","reported_seq":12569,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.982168+0000","last_change":"2024-09-14T22:53:51.942879+0000","last_active":"2024-09-14T22:53:57.982168+0000","last_peered":"2024-09-14T22:53:57.982168+0000","last_clean":"2024-09-14T22:53:57.982168+0000","last_became_active":"2024-09-14T22:53:51.942267+0000","last_became_peered":"2024-09-14T22:53:51.942267+0000","last_unstale":"2024-09-14T22:53:57.982168+0000","last_undegraded":"2024-09-14T22:53:57.982168+0000","last_fullsized":"2024-09-14T22:53:57.982168+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:59:59.619901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":28826682,"num_objects":61,"num_object_clones":9,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":39,"num_read":13984,"num_read_kb":35084,"num_write":566,"num_write_kb":17346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":113,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.995254+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:53:57.995254+0000","last_peered":"2024-09-14T22:53:57.995254+0000","last_clean":"2024-09-14T22:53:57.995254+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:53:57.995254+0000","last_undegraded":"2024-09-14T22:53:57.995254+0000","last_fullsized":"2024-09-14T22:53:57.995254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'175","reported_seq":13563,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.733929+0000","last_change":"2024-09-14T22:53:35.869981+0000","last_active":"2024-09-14T22:53:58.733929+0000","last_peered":"2024-09-14T22:53:58.733929+0000","last_clean":"2024-09-14T22:53:58.733929+0000","last_became_active":"2024-09-14T22:53:35.868554+0000","last_became_peered":"2024-09-14T22:53:35.868554+0000","last_unstale":"2024-09-14T22:53:58.733929+0000","last_undegraded":"2024-09-14T22:53:58.733929+0000","last_fullsized":"2024-09-14T22:53:58.733929+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:56:59.688670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":65775013,"num_objects":72,"num_object_clones":12,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":43,"num_read":11897,"num_read_kb":38265,"num_write":152,"num_write_kb":9814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":3794613,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"65'257","reported_seq":4750,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.982313+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:53:57.982313+0000","last_peered":"2024-09-14T22:53:57.982313+0000","last_clean":"2024-09-14T22:53:57.982313+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:53:57.982313+0000","last_undegraded":"2024-09-14T22:53:57.982313+0000","last_fullsized":"2024-09-14T22:53:57.982313+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:07:23.863368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":43383469,"num_objects":68,"num_object_clones":13,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":46,"num_read":5831,"num_read_kb":52610,"num_write":225,"num_write_kb":29279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":115,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.001474+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:53:58.001474+0000","last_peered":"2024-09-14T22:53:58.001474+0000","last_clean":"2024-09-14T22:53:58.001474+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:53:58.001474+0000","last_undegraded":"2024-09-14T22:53:58.001474+0000","last_fullsized":"2024-09-14T22:53:58.001474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"68'149","reported_seq":836,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.734033+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:53:58.734033+0000","last_peered":"2024-09-14T22:53:58.734033+0000","last_clean":"2024-09-14T22:53:58.734033+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:53:58.734033+0000","last_undegraded":"2024-09-14T22:53:58.734033+0000","last_fullsized":"2024-09-14T22:53:58.734033+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:10:11.243876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":8767985,"num_objects":36,"num_object_clones":3,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":23,"num_read":904,"num_read_kb":7480,"num_write":173,"num_write_kb":10559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.981979+0000","last_change":"2024-09-14T22:53:32.745426+0000","last_active":"2024-09-14T22:53:57.981979+0000","last_peered":"2024-09-14T22:53:57.981979+0000","last_clean":"2024-09-14T22:53:57.981979+0000","last_became_active":"2024-09-14T22:53:32.745120+0000","last_became_peered":"2024-09-14T22:53:32.745120+0000","last_unstale":"2024-09-14T22:53:57.981979+0000","last_undegraded":"2024-09-14T22:53:57.981979+0000","last_fullsized":"2024-09-14T22:53:57.981979+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:06:49.621943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"69'395","reported_seq":6170,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.731912+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:53:58.731912+0000","last_peered":"2024-09-14T22:53:58.731912+0000","last_clean":"2024-09-14T22:53:58.731912+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:53:58.731912+0000","last_undegraded":"2024-09-14T22:53:58.731912+0000","last_fullsized":"2024-09-14T22:53:58.731912+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:13:09.030404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":11719457,"num_objects":50,"num_object_clones":7,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":31,"num_read":7477,"num_read_kb":10998,"num_write":544,"num_write_kb":1690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":113,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.991598+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:53:57.991598+0000","last_peered":"2024-09-14T22:53:57.991598+0000","last_clean":"2024-09-14T22:53:57.991598+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:53:57.991598+0000","last_undegraded":"2024-09-14T22:53:57.991598+0000","last_fullsized":"2024-09-14T22:53:57.991598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"69'163","reported_seq":4870,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.132102+0000","last_change":"2024-09-14T22:53:32.825251+0000","last_active":"2024-09-14T22:53:59.132102+0000","last_peered":"2024-09-14T22:53:59.132102+0000","last_clean":"2024-09-14T22:53:59.132102+0000","last_became_active":"2024-09-14T22:53:32.746422+0000","last_became_peered":"2024-09-14T22:53:32.746422+0000","last_unstale":"2024-09-14T22:53:59.132102+0000","last_undegraded":"2024-09-14T22:53:59.132102+0000","last_fullsized":"2024-09-14T22:53:59.132102+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:58:32.046352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":22352282,"num_objects":56,"num_object_clones":5,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":41,"num_read":4573,"num_read_kb":11120,"num_write":126,"num_write_kb":18004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":57590,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":114,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.001488+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:53:58.001488+0000","last_peered":"2024-09-14T22:53:58.001488+0000","last_clean":"2024-09-14T22:53:58.001488+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:53:58.001488+0000","last_undegraded":"2024-09-14T22:53:58.001488+0000","last_fullsized":"2024-09-14T22:53:58.001488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":264,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.991680+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:53:57.991680+0000","last_peered":"2024-09-14T22:53:57.991680+0000","last_clean":"2024-09-14T22:53:57.991680+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:53:57.991680+0000","last_undegraded":"2024-09-14T22:53:57.991680+0000","last_fullsized":"2024-09-14T22:53:57.991680+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"69'307","reported_seq":5143,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.121301+0000","last_change":"2024-09-14T22:53:48.924907+0000","last_active":"2024-09-14T22:53:59.121301+0000","last_peered":"2024-09-14T22:53:59.121301+0000","last_clean":"2024-09-14T22:53:59.121301+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:53:59.121301+0000","last_undegraded":"2024-09-14T22:53:59.121301+0000","last_fullsized":"2024-09-14T22:53:59.121301+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:37:21.374284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":18486965,"num_objects":68,"num_object_clones":13,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":43,"num_read":6372,"num_read_kb":19963,"num_write":282,"num_write_kb":12007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"68'124","reported_seq":603,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.732200+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:53:58.732200+0000","last_peered":"2024-09-14T22:53:58.732200+0000","last_clean":"2024-09-14T22:53:58.732200+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:53:58.732200+0000","last_undegraded":"2024-09-14T22:53:58.732200+0000","last_fullsized":"2024-09-14T22:53:58.732200+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:47:00.229335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":6989713,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":40,"num_read":414,"num_read_kb":1492,"num_write":82,"num_write_kb":1237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"69'272","reported_seq":1051,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.735186+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:53:58.735186+0000","last_peered":"2024-09-14T22:53:58.735186+0000","last_clean":"2024-09-14T22:53:58.735186+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:53:58.735186+0000","last_undegraded":"2024-09-14T22:53:58.735186+0000","last_fullsized":"2024-09-14T22:53:58.735186+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:15:25.185135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":25045056,"num_objects":48,"num_object_clones":5,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":32,"num_read":1745,"num_read_kb":5725,"num_write":308,"num_write_kb":3595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"67'187","reported_seq":1652,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.132633+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:53:58.132633+0000","last_peered":"2024-09-14T22:53:58.132633+0000","last_clean":"2024-09-14T22:53:58.132633+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:53:58.132633+0000","last_undegraded":"2024-09-14T22:53:58.132633+0000","last_fullsized":"2024-09-14T22:53:58.132633+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:11:26.841286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":12839551,"num_objects":55,"num_object_clones":8,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":41,"num_read":1860,"num_read_kb":14659,"num_write":150,"num_write_kb":8049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"67'187","reported_seq":3790,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.001578+0000","last_change":"2024-09-14T22:53:35.869109+0000","last_active":"2024-09-14T22:53:58.001578+0000","last_peered":"2024-09-14T22:53:58.001578+0000","last_clean":"2024-09-14T22:53:58.001578+0000","last_became_active":"2024-09-14T22:53:35.868711+0000","last_became_peered":"2024-09-14T22:53:35.868711+0000","last_unstale":"2024-09-14T22:53:58.001578+0000","last_undegraded":"2024-09-14T22:53:58.001578+0000","last_fullsized":"2024-09-14T22:53:58.001578+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:47.242789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":17229184,"num_objects":57,"num_object_clones":5,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":39,"num_read":3566,"num_read_kb":10286,"num_write":156,"num_write_kb":3526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1667203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"68'164","reported_seq":2932,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.991891+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:53:57.991891+0000","last_peered":"2024-09-14T22:53:57.991891+0000","last_clean":"2024-09-14T22:53:57.991891+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:53:57.991891+0000","last_undegraded":"2024-09-14T22:53:57.991891+0000","last_fullsized":"2024-09-14T22:53:57.991891+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:03:57.273511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":11194078,"num_objects":57,"num_object_clones":6,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":43,"num_read":2878,"num_read_kb":11321,"num_write":157,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"69'181","reported_seq":1534,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.156323+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:53:59.156323+0000","last_peered":"2024-09-14T22:53:59.156323+0000","last_clean":"2024-09-14T22:53:59.156323+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:53:59.156323+0000","last_undegraded":"2024-09-14T22:53:59.156323+0000","last_fullsized":"2024-09-14T22:53:59.156323+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:17:00.759941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":15600421,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":33,"num_read":1903,"num_read_kb":5426,"num_write":154,"num_write_kb":3137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"67'145","reported_seq":3613,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.991837+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:53:57.991837+0000","last_peered":"2024-09-14T22:53:57.991837+0000","last_clean":"2024-09-14T22:53:57.991837+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:53:57.991837+0000","last_undegraded":"2024-09-14T22:53:57.991837+0000","last_fullsized":"2024-09-14T22:53:57.991837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:00:01.887367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":23910680,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":33,"num_read":3484,"num_read_kb":18813,"num_write":130,"num_write_kb":10737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"69'205","reported_seq":8096,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:58.734438+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:53:58.734438+0000","last_peered":"2024-09-14T22:53:58.734438+0000","last_clean":"2024-09-14T22:53:58.734438+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:53:58.734438+0000","last_undegraded":"2024-09-14T22:53:58.734438+0000","last_fullsized":"2024-09-14T22:53:58.734438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:43.580778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":40464331,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":36,"num_read":9093,"num_read_kb":24360,"num_write":166,"num_write_kb":16146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"69'235","reported_seq":4119,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:59.131848+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:53:59.131848+0000","last_peered":"2024-09-14T22:53:59.131848+0000","last_clean":"2024-09-14T22:53:59.131848+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:53:59.131848+0000","last_undegraded":"2024-09-14T22:53:59.131848+0000","last_fullsized":"2024-09-14T22:53:59.131848+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:25:44.212967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":46710648,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":4327,"num_read_kb":21084,"num_write":301,"num_write_kb":19228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"61'238","reported_seq":939,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.991702+0000","last_change":"2024-09-14T22:53:38.883182+0000","last_active":"2024-09-14T22:53:57.991702+0000","last_peered":"2024-09-14T22:53:57.991702+0000","last_clean":"2024-09-14T22:53:57.991702+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:53:57.991702+0000","last_undegraded":"2024-09-14T22:53:57.991702+0000","last_fullsized":"2024-09-14T22:53:57.991702+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:21:33.215215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":38375544,"num_objects":48,"num_object_clones":5,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":35,"num_read":1143,"num_read_kb":8156,"num_write":303,"num_write_kb":8061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"67'226","reported_seq":2607,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-14T22:53:57.991877+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:53:57.991877+0000","last_peered":"2024-09-14T22:53:57.991877+0000","last_clean":"2024-09-14T22:53:57.991877+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:53:57.991877+0000","last_undegraded":"2024-09-14T22:53:57.991877+0000","last_fullsized":"2024-09-14T22:53:57.991877+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:13.502722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":47003855,"num_objects":53,"num_object_clones":5,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":42,"num_read":3036,"num_read_kb":3007,"num_write":167,"num_write_kb":1802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1009539501,"num_objects":1723,"num_object_clones":242,"num_object_copies":3446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1723,"num_whiteouts":1138,"num_read":169458,"num_read_kb":638844,"num_write":7481,"num_write_kb":358355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":17845622,"num_keys_recovered":18,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":502054912,"data_stored":592845954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7220,"ondisk_log_size":7220,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102192,"kb_used_data":66848,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94269648,"statfs":{"total":96636764160,"available":96532119552,"internally_reserved":0,"allocated":68452352,"data_stored":81171776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69548,"kb_used_data":38360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302292,"statfs":{"total":96636764160,"available":96565547008,"internally_reserved":0,"allocated":39280640,"data_stored":46154047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85384,"kb_used_data":50040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94286456,"statfs":{"total":96636764160,"available":96549330944,"internally_reserved":0,"allocated":51240960,"data_stored":57297187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":155952,"kb_used_data":116392,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94215888,"statfs":{"total":96636764160,"available":96477069312,"internally_reserved":0,"allocated":119185408,"data_stored":139364112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71456,"kb_used_data":36112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94300384,"statfs":{"total":96636764160,"available":96563593216,"internally_reserved":0,"allocated":36978688,"data_stored":39684114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62236,"kb_used_data":31052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309604,"statfs":{"total":96636764160,"available":96573034496,"internally_reserved":0,"allocated":31797248,"data_stored":42038812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103928,"kb_used_data":68584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94267912,"statfs":{"total":96636764160,"available":96530341888,"internally_reserved":0,"allocated":70230016,"data_stored":89395727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":129900,"kb_used_data":90340,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94241940,"statfs":{"total":96636764160,"available":96503746560,"internally_reserved":0,"allocated":92508160,"data_stored":101813689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91111424,"data_stored":100861729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69423104,"data_stored":89034116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30994432,"data_stored":41677220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36175872,"data_stored":39322522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":118382592,"data_stored":139002520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":50434048,"data_stored":56935576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":38477824,"data_stored":45792455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":67055616,"data_stored":80219816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:54:01.861 INFO:tasks.thrashosds.thrasher:Setting 3.7 to [7, 6] 2024-09-14T22:54:01.861 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.7', '7', '6'] 2024-09-14T22:54:01.861 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.7 7 6 2024-09-14T22:54:01.862 INFO:teuthology.orchestra.run.smithi076.stdout:2731 read 0x178d46 thru 0x182c81 (0x9f3c bytes) 2024-09-14T22:54:01.862 INFO:teuthology.orchestra.run.smithi076.stdout:2732 write 0x63d917 thru 0x642206 (0x48f0 bytes) 2024-09-14T22:54:01.870 INFO:teuthology.orchestra.run.smithi076.stdout:2733 trunc from 0x32a0a6f to 0xaaeb5bf 2024-09-14T22:54:01.875 INFO:teuthology.orchestra.run.smithi076.stdout:2734 writesame 0x38081d1 thru 0x380fc1d data_size 0x1(0x7a4d bytes) 2024-09-14T22:54:01.890 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:01.891 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:01.893 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.893 INFO:teuthology.orchestra.run.smithi076.stdout:2739 read 0x4f46c61 thru 0x4f53a8a (0xce2a bytes) 2024-09-14T22:54:01.902 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:01.907 INFO:teuthology.orchestra.run.smithi076.stdout:2740 trunc from 0xaaeb5bf to 0x2f5364 2024-09-14T22:54:01.925 INFO:teuthology.orchestra.run.smithi076.stdout:2741 trunc from 0x2f5364 to 0x23e3f56 2024-09-14T22:54:01.925 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:01.924+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:01.929 INFO:teuthology.orchestra.run.smithi076.stdout:2742 punch from 0x47c60b to 0x4892c2, (0xccb7 bytes) 2024-09-14T22:54:01.930 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.942 INFO:teuthology.orchestra.run.smithi076.stdout:2744 trunc from 0x23e3f56 to 0xce9070b 2024-09-14T22:54:01.954 INFO:teuthology.orchestra.run.smithi076.stdout:2746 writesame 0xedd2af0 thru 0xeddf0c2 data_size 0x33(0xc5d3 bytes) 2024-09-14T22:54:01.954 INFO:teuthology.orchestra.run.smithi076.stdout:2750 write 0x3c387c3 thru 0x3c3bdb2 (0x35f0 bytes) 2024-09-14T22:54:01.957 INFO:teuthology.orchestra.run.smithi076.stdout:2751 read 0x4871f4b thru 0x487577f (0x3835 bytes) 2024-09-14T22:54:01.958 INFO:teuthology.orchestra.run.smithi076.stdout:2752 trunc from 0xeddf0c3 to 0xe13ca04 2024-09-14T22:54:01.963 INFO:teuthology.orchestra.run.smithi076.stdout:2753 read 0xa8bd5f3 thru 0xa8c8b0e (0xb51c bytes) 2024-09-14T22:54:01.964 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.964 INFO:teuthology.orchestra.run.smithi076.stdout:2756 punch from 0x79ead0 to 0x79f217, (0x747 bytes) 2024-09-14T22:54:01.968 INFO:teuthology.orchestra.run.smithi076.stdout:2757 trunc from 0xe13ca04 to 0xa99c41b 2024-09-14T22:54:01.981 INFO:teuthology.orchestra.run.smithi076.stdout:2758 write 0xbfcafdf thru 0xbfdaac8 (0xfaea bytes) 2024-09-14T22:54:01.985 INFO:teuthology.orchestra.run.smithi076.stdout:2759 punch from 0x7a05ea6 to 0x7a121fe, (0xc358 bytes) 2024-09-14T22:54:01.985 INFO:teuthology.orchestra.run.smithi076.stdout:2760 read 0x56061d9 thru 0x561571a (0xf542 bytes) 2024-09-14T22:54:01.986 INFO:teuthology.orchestra.run.smithi076.stdout:2761 trunc from 0xbfdaac9 to 0xb922a67 2024-09-14T22:54:01.993 INFO:teuthology.orchestra.run.smithi076.stdout:2763 writesame 0x80bde7d thru 0x80c1776 data_size 0x66(0x38fa bytes) 2024-09-14T22:54:01.997 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:01.997 INFO:teuthology.orchestra.run.smithi076.stdout:2766 read 0x54e4a1c thru 0x54e9866 (0x4e4b bytes) 2024-09-14T22:54:02.005 INFO:teuthology.orchestra.run.smithi076.stdout:2767 trunc from 0xb922a67 to 0x4a0c2bb 2024-09-14T22:54:02.018 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:02.017+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:02.029 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:02.029 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:02.031 INFO:teuthology.orchestra.run.smithi076.stdout:2768 write 0xb572693 thru 0xb575959 (0x32c7 bytes) 2024-09-14T22:54:02.032 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:02.032 INFO:teuthology.orchestra.run.smithi076.stdout:2770 write 0xaf0b022 thru 0xaf19feb (0xefca bytes) 2024-09-14T22:54:02.034 INFO:teuthology.orchestra.run.smithi076.stdout:2772 write 0x5db3f56 thru 0x5db98a7 (0x5952 bytes) 2024-09-14T22:54:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:2776 read 0x8bd1dda thru 0x8bd96d6 (0x78fd bytes) 2024-09-14T22:54:02.038 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:02.039 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:02.043 INFO:teuthology.orchestra.run.smithi076.stdout:2779 writesame 0xbc6dce9 thru 0xbc7b718 data_size 0x8(0xda30 bytes) 2024-09-14T22:54:02.045 INFO:teuthology.orchestra.run.smithi076.stdout:2781 write 0x1f8bbae thru 0x1f8e4e4 (0x2937 bytes) 2024-09-14T22:54:02.045 INFO:teuthology.orchestra.run.smithi076.stdout:2783 read 0x3596b0c thru 0x35a1ddb (0xb2d0 bytes) 2024-09-14T22:54:02.046 INFO:teuthology.orchestra.run.smithi076.stdout:2786 write 0x4f6c823 thru 0x4f78f8f (0xc76d bytes) 2024-09-14T22:54:02.049 INFO:teuthology.orchestra.run.smithi076.stdout:2787 write 0x2638881 thru 0x263bebc (0x363c bytes) 2024-09-14T22:54:02.054 INFO:teuthology.orchestra.run.smithi076.stdout:2788 trunc from 0xbc7b719 to 0xe1909b6 2024-09-14T22:54:02.059 INFO:teuthology.orchestra.run.smithi076.stdout:2789 write 0x33a522e thru 0x33b2472 (0xd245 bytes) 2024-09-14T22:54:02.061 INFO:teuthology.orchestra.run.smithi076.stdout:2790 write 0xd829ef8 thru 0xd82eff2 (0x50fb bytes) 2024-09-14T22:54:02.062 INFO:teuthology.orchestra.run.smithi076.stdout:2791 punch from 0x8c71a7 to 0x8c9193, (0x1fec bytes) 2024-09-14T22:54:02.062 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:02.062 INFO:teuthology.orchestra.run.smithi076.stdout:2795 read 0x4ebe931 thru 0x4ece0dc (0xf7ac bytes) 2024-09-14T22:54:02.063 INFO:teuthology.orchestra.run.smithi076.stdout:2796 clone 27 order 19 su 8192 sc 3 2024-09-14T22:54:02.119 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:02.117+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:02.148 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:02.148 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:02.157 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:02.220 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:02.219+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:02.301 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:02.301 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:02.309 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:02.354 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:02.353+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:02.413 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:02.414 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:02.423 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:02.424 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:02.420+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:02.523 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:02.522+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:02.567 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:02.567 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:02.575 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:02.659 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:02.658+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:02.719 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:02.719 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:02.719 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:02.728 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:02.728 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:02.726+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:02.826 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:02.824+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:02.842 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:02.842 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:02.859 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:02.859 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:02.874 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:02.874 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:02.876 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:02.876 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:02.884 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:02.893 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:02.893 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:02.911 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:02.911 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:02.927 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:02.926+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:02.928 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:02.928 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:02.945 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:02.945 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:02.962 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:02.962 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:02.989 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:02.989 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:02.989 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:02.990 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:02.998 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:03.019 INFO:teuthology.orchestra.run.smithi076.stderr:set 3.7 pg_upmap mapping to [7,6] 2024-09-14T22:54:03.022 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:54:03.028 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:03.026+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:03.092 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone26 from 0xe1909b6 (overlap 0x2f5364) to 0x2a97da 2024-09-14T22:54:03.129 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:03.127+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:03.130 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:03.130 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:03.130 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:03.130 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:03.130 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:03.140 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:03.162 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T22:54:03.164 INFO:teuthology.orchestra.run.smithi076.stdout:2797 punch from 0x1871139 to 0x18805f9, (0xf4c0 bytes) 2024-09-14T22:54:03.166 INFO:teuthology.orchestra.run.smithi076.stdout:2799 write 0xb8fee14 thru 0xb90b8b8 (0xcaa5 bytes) 2024-09-14T22:54:03.197 INFO:teuthology.orchestra.run.smithi076.stdout:2801 read 0x8931adc thru 0x8938fed (0x7512 bytes) 2024-09-14T22:54:03.203 INFO:teuthology.orchestra.run.smithi076.stdout:2803 trunc from 0xe1909b6 to 0xa164f55 2024-09-14T22:54:03.229 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:03.228+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:03.251 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:03.251 INFO:teuthology.orchestra.run.smithi076.stdout:2805 read 0x9acc035 thru 0x9ad41af (0x817b bytes) 2024-09-14T22:54:03.251 INFO:teuthology.orchestra.run.smithi076.stdout:2808 writesame 0x47e246 thru 0x485398 data_size 0x1(0x7153 bytes) 2024-09-14T22:54:03.290 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:03.290 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383687053000003, 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:54:03.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.295 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383666517000002, 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:54:03.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.297 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:03.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:03.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:54:03.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:54:03.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383636518000003, 2024-09-14T22:54:03.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:54:03.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.307 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383605365000001, 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.308 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:03.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.312 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383582709000002, 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:54:03.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383562208000001, 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.317 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:03.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.321 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383542447000004, 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:54:03.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:54:03.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383521662, 2024-09-14T22:54:03.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:03.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:54:03.330 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383489249999997, 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.331 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:54:03.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:03.335 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383467850999999, 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383406452000003, 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.340 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:54:03.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:54:03.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:54:03.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383366539000001, 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.344 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:03.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:54:03.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:03.348 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383336782999997, 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383306548, 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.353 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:54:03.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383281809000003, 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.358 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:03.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:54:03.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383258378999997, 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.362 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:54:03.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383162642999999, 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:54:03.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:54:03.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383133721, 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.371 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:03.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:54:03.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.383110395999999, 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:54:03.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.250412653000001, 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:03.380 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:54:03.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:03.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:54:03.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:54:03.385 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.385 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.385 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.385 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:03.385 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:03.385 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:03.387 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:03.387 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:03.387 INFO:teuthology.orchestra.run.smithi076.stdout:2813 write 0xe11206c thru 0xe1198a7 (0x783c bytes) 2024-09-14T22:54:03.387 INFO:teuthology.orchestra.run.smithi076.stdout:2815 trunc from 0xe1198a8 to 0x340976c 2024-09-14T22:54:03.388 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:03.388 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:03.385+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:03.388 INFO:teuthology.orchestra.run.smithi076.stdout:2817 write 0xb3eb3a9 thru 0xb3ee10b (0x2d63 bytes) 2024-09-14T22:54:03.392 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:03.392 INFO:teuthology.orchestra.run.smithi076.stdout:2819 read 0x817587f thru 0x8177fd8 (0x275a bytes) 2024-09-14T22:54:03.392 INFO:teuthology.orchestra.run.smithi076.stdout:2822 writesame 0x353487 thru 0x3567fe data_size 0x1b(0x3378 bytes) 2024-09-14T22:54:03.431 INFO:teuthology.orchestra.run.smithi076.stdout:2823 write 0x3b74cfd thru 0x3b7b70c (0x6a10 bytes) 2024-09-14T22:54:03.435 INFO:teuthology.orchestra.run.smithi076.stdout:2824 write 0x9d65b8d thru 0x9d73761 (0xdbd5 bytes) 2024-09-14T22:54:03.487 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:03.486+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:03.495 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153281 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'291, mlcod=72'289)", 2024-09-14T22:54:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.059274362999999997, 2024-09-14T22:54:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059296640999999997, 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435411+0000", 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5589999999999999e-06 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435419+0000", 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.909e-06 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435423+0000", 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5140000000000003e-06 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435458+0000", 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965000000000002e-05 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435470+0000", 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1559999999999999e-05 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 1 2024-09-14T22:54:03.499 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:03.500 INFO:teuthology.orchestra.run.smithi076.stdout:2825 trunc from 0xb3ee10c to 0x99efc83 2024-09-14T22:54:03.506 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:03.521 INFO:teuthology.orchestra.run.smithi076.stdout:2826 write 0xa47c1d7 thru 0xa48bc1c (0xfa46 bytes) 2024-09-14T22:54:03.529 INFO:teuthology.orchestra.run.smithi076.stdout:2827 write 0x300a59a thru 0x301952a (0xef91 bytes) 2024-09-14T22:54:03.561 INFO:teuthology.orchestra.run.smithi076.stdout:2828 read 0x56c0cee thru 0x56c6e09 (0x611c bytes) 2024-09-14T22:54:03.569 INFO:teuthology.orchestra.run.smithi076.stdout:2830 trunc from 0xa48bc1d to 0x439e15d 2024-09-14T22:54:03.580 INFO:teuthology.orchestra.run.smithi076.stdout:2831 write 0x5aa4ec thru 0x5ab86a (0x137f bytes) 2024-09-14T22:54:03.582 INFO:teuthology.orchestra.run.smithi076.stdout:2832 writesame 0x155a8e6 thru 0x1560963 data_size 0x2e(0x607e bytes) 2024-09-14T22:54:03.588 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:03.586+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:03.623 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:03.623 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:03.623 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:03.623 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:03.624 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:03.633 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:03.633 INFO:teuthology.orchestra.run.smithi076.stdout:2833 write 0x89b4071 thru 0x89c2c5c (0xebec bytes) 2024-09-14T22:54:03.647 INFO:teuthology.orchestra.run.smithi076.stdout:2834 write 0x96b23a6 thru 0x96bff33 (0xdb8e bytes) 2024-09-14T22:54:03.657 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:03.665 INFO:teuthology.orchestra.run.smithi076.stdout:2836 trunc from 0x96bff34 to 0x2fbad6e 2024-09-14T22:54:03.689 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:03.687+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:03.711 INFO:teuthology.orchestra.run.smithi076.stdout:2837 punch from 0x185a959 to 0x1861ec3, (0x756a bytes) 2024-09-14T22:54:03.718 INFO:teuthology.orchestra.run.smithi076.stdout:2838 trunc from 0x2fbad6e to 0x972496f 2024-09-14T22:54:03.725 INFO:teuthology.orchestra.run.smithi076.stdout:2839 read 0x3d88119 thru 0x3d894b6 (0x139e bytes) 2024-09-14T22:54:03.726 INFO:teuthology.orchestra.run.smithi076.stdout:2841 punch from 0x2e2fc41 to 0x2e3da34, (0xddf3 bytes) 2024-09-14T22:54:03.726 INFO:teuthology.orchestra.run.smithi076.stdout:2842 read 0x90d67c6 thru 0x90db13c (0x4977 bytes) 2024-09-14T22:54:03.726 INFO:teuthology.orchestra.run.smithi076.stdout:2845 read 0x961a632 thru 0x9623e39 (0x9808 bytes) 2024-09-14T22:54:03.732 INFO:teuthology.orchestra.run.smithi076.stdout:2846 trunc from 0x972496f to 0x3d87df1 2024-09-14T22:54:03.738 INFO:teuthology.orchestra.run.smithi076.stdout:2847 read 0xe9a171 thru 0xe9b667 (0x14f7 bytes) 2024-09-14T22:54:03.754 INFO:teuthology.orchestra.run.smithi076.stdout:2849 write 0xc1a8a95 thru 0xc1b2a21 (0x9f8d bytes) 2024-09-14T22:54:03.759 INFO:teuthology.orchestra.run.smithi076.stdout:2851 read 0x2825848 thru 0x282a8af (0x5068 bytes) 2024-09-14T22:54:03.761 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:03.761 INFO:teuthology.orchestra.run.smithi076.stdout:2854 writesame 0x3ab01e0 thru 0x3ab7206 data_size 0x1(0x7027 bytes) 2024-09-14T22:54:03.778 INFO:teuthology.orchestra.run.smithi076.stdout:2856 write 0x6644da0 thru 0x66536df (0xe940 bytes) 2024-09-14T22:54:03.785 INFO:teuthology.orchestra.run.smithi076.stdout:2858 punch from 0x6c5eb82 to 0x6c6dbf1, (0xf06f bytes) 2024-09-14T22:54:03.789 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:03.788+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:03.790 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:03.790 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:03.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:03.790 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116113 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5611520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.733968708999999, 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053031104000000003, 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.791 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116113 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053986+0000", 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010658 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053989+0000", 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3579999999999999e-06 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053998+0000", 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8319999999999995e-06 2024-09-14T22:54:03.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054004+0000", 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3579999999999999e-06 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106732+0000", 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052727505000000001 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106816+0000", 2024-09-14T22:54:03.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4482999999999997e-05 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106910+0000", 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3987999999999995e-05 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.733833011000002, 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.796 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:54:03.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.733637919, 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:54:03.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.733485129000002, 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:54:03.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:54:03.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.733280106999999, 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.733009441, 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:54:03.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:54:03.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.732747281000002, 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:54:03.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.732588429, 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:54:03.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:03.829 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.732401632999998, 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:03.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:54:03.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.732199388000002, 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:03.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:54:03.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.73199103, 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.839 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:54:03.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.731677312999999, 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.843 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:54:03.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:54:03.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.731471550999998, 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.848 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:54:03.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.731302674999998, 2024-09-14T22:54:03.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:03.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:54:03.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.856 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.731031981000001, 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:54:03.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.730882705999999, 2024-09-14T22:54:03.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.730641464000001, 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:54:03.866 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:54:03.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.870 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.421951432, 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:03.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:54:03.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153281 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'291, mlcod=72'289)", 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.35244196999999999, 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060467871999999999, 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.877 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435411+0000", 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5589999999999999e-06 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435419+0000", 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.909e-06 2024-09-14T22:54:03.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435423+0000", 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5140000000000003e-06 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435458+0000", 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965000000000002e-05 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435470+0000", 2024-09-14T22:54:03.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1559999999999999e-05 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495827+0000", 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060356819999999999 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495874+0000", 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7003999999999999e-05 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153282 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'292, mlcod=72'289)", 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.35181025399999999, 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060246112999999997, 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436038+0000", 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:03.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436042+0000", 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8920000000000002e-06 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495165+0000", 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059122945000000003 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495179+0000", 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4185000000000001e-05 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495346+0000", 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016677400000000001 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495373+0000", 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7172e-05 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496245+0000", 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00087206100000000004 2024-09-14T22:54:03.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496284+0000", 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8525000000000001e-05 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:03.886 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:03.888 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:03.888 INFO:teuthology.orchestra.run.smithi076.stdout:2859 write 0xd10603b thru 0xd10dfc1 (0x7f87 bytes) 2024-09-14T22:54:03.889 INFO:teuthology.orchestra.run.smithi076.stdout:2861 write 0x5d84672 thru 0x5d89240 (0x4bcf bytes) 2024-09-14T22:54:03.889 INFO:teuthology.orchestra.run.smithi076.stdout:2862 trunc from 0xd10dfc2 to 0x377690e 2024-09-14T22:54:03.889 INFO:teuthology.orchestra.run.smithi076.stdout:2863 write 0x4502f88 thru 0x4504626 (0x169f bytes) 2024-09-14T22:54:03.889 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:03.889 INFO:teuthology.orchestra.run.smithi076.stdout:2865 punch from 0x675e37 to 0x6841eb, (0xe3b4 bytes) 2024-09-14T22:54:03.889 INFO:teuthology.orchestra.run.smithi076.stdout:2867 read 0x3e62f9b thru 0x3e65983 (0x29e9 bytes) 2024-09-14T22:54:03.889 INFO:teuthology.orchestra.run.smithi076.stdout:2868 read 0x1733589 thru 0x1735dfe (0x2876 bytes) 2024-09-14T22:54:03.889 INFO:teuthology.orchestra.run.smithi076.stdout:2869 clone 28 order 25 su 4194304 sc 12 2024-09-14T22:54:03.890 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:03.889+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:03.992 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:03.990+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:03.993 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:03.993 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:04.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:04.018 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:54:04.093 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:04.092+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:04.097 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone27 from 0x4504627 (overlap 0x2fbad6e) to 0x4c5e24 2024-09-14T22:54:04.150 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:04.150 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:04.150 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:04.150 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:04.150 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:04.159 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:04.202 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:04.201+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:04.283 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.384583970000001, 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:54:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:54:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127349 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4120576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.758894414, 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053205084, 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127349 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7000000000000005e-07 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521644+0000", 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521715+0000", 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0736999999999997e-05 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521725+0000", 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0237999999999999e-05 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574705+0000", 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052979928000000003 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574768+0000", 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2618000000000003e-05 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574846+0000", 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8143e-05 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.758789851, 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:54:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.758701858999999, 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:54:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.758467941999999, 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.758287047, 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:54:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:54:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:54:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.757987543999999, 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:54:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127361 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.757667397999999, 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053260788000000003, 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127361 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522869+0000", 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522872+0000", 2024-09-14T22:54:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.066e-06 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522879+0000", 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8110000000000003e-06 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522885+0000", 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.003e-06 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575959+0000", 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073515000000002 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576037+0000", 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7991999999999995e-05 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576129+0000", 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1772000000000002e-05 2024-09-14T22:54:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.757641333999999, 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:54:04.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.757277605999999, 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.330 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:54:04.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.334 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.756687989, 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.335 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:54:04.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.339 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.756485752, 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.340 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:54:04.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:54:04.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.756295941000001, 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:54:04.344 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:04.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:54:04.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:54:04.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:54:04.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:54:04.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.755983938, 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:04.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:54:04.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:54:04.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:54:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:54:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.755741450999999, 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:54:04.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:54:04.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.365 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:04.366 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.755582488999998, 2024-09-14T22:54:04.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:54:04.366 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.366 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.366 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.366 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:54:04.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.370 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.754948218999999, 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.371 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:54:04.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.754734109000001, 2024-09-14T22:54:04.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:54:04.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:54:04.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.754597639, 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:54:04.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.754345159, 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.384 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:54:04.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:54:04.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.388 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.389 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.389 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:04.390 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:04.391 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:04.388+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:04.490 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:04.489+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:04.575 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:04.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:04.576 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:04.576 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:04.584 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:04.629 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:04.628+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:04.692 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:04.691+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:04.712 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:04.712 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:04.722 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:04.806 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:04.805+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:04.867 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:04.867 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:04.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:04.867 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:04.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.867 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.867 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:04.867 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.783210171, 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:54:04.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:54:04.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:54:04.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.782992274999998, 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.780308628999997, 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:04.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:54:04.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.780150214000003, 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.881 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:04.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.885 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.780122947999999, 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:54:04.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:54:04.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.779915807000002, 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:54:04.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.894 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.779881543000002, 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.898 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.778249496999997, 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:04.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:54:04.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.778224885, 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.904 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:54:04.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.778090599000002, 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:54:04.908 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:54:04.912 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.777040890999999, 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.913 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:04.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.776938706000003, 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:54:04.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:54:04.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.922 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.776914802, 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:54:04.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:54:04.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.776881281999998, 2024-09-14T22:54:04.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:04.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:54:04.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.776726635999999, 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.932 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.936 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.776685462000003, 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:54:04.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.776657301999997, 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.941 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:54:04.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.945 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.776467910999997, 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.946 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:54:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:54:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.860378083999997, 2024-09-14T22:54:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:54:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:54:04.954 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 40.860358253000001, 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:04.955 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:04.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:04.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:04.959 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:04.961 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:04.961 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:04.959+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:04.974 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:05.062 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:05.061+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:05.098 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:05.106 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:155997 3.7 3.8ad265a7 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.418719+0000", 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.68734694900000004, 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.68737287800000002, 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "waiting for peered", 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:05.107 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 155997 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.418719+0000", 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.418719+0000", 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.418719+0000", 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4600000000000004e-07 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.418723+0000", 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7479999999999999e-06 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.418730+0000", 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2420000000000003e-06 2024-09-14T22:54:05.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.418735+0000", 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0669999999999999e-06 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.418784+0000", 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9110000000000002e-05 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.418794+0000", 2024-09-14T22:54:05.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3689999999999996e-06 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:155997 3.7 3.8ad265a7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.281113+0000", 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.82495205299999996, 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.82501727499999999, 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "waiting for peered", 2024-09-14T22:54:05.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 155997 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.281113+0000", 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.281113+0000", 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.281114+0000", 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7000000000000001e-07 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.281115+0000", 2024-09-14T22:54:05.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.514e-06 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.281119+0000", 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4690000000000002e-06 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.281121+0000", 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4540000000000001e-06 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:05.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.281162+0000", 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1159999999999999e-05 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.281169+0000", 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3990000000000001e-06 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:155997 3.7 3.8ad265a7 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:05.115 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.426177+0000", 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.67988829500000003, 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.67998673799999998, 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "waiting for peered", 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 155997 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.426177+0000", 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.426177+0000", 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.426178+0000", 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1500000000000004e-07 2024-09-14T22:54:05.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.426181+0000", 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3069999999999998e-06 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.426186+0000", 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0340000000000004e-06 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:05.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.426193+0000", 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3790000000000003e-06 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.426247+0000", 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4456000000000002e-05 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.426256+0000", 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2869999999999993e-06 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:05.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:158052 3.7 3.27814be7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.525185+0000", 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.58088002900000002, 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.58101027800000005, 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "waiting for peered", 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 158052 2024-09-14T22:54:05.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.525185+0000", 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.525185+0000", 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:05.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.525186+0000", 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-07 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.525190+0000", 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0500000000000002e-06 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.525199+0000", 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2989999999999997e-06 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.525203+0000", 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5329999999999996e-06 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.525244+0000", 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1359999999999997e-05 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.525253+0000", 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4030000000000007e-06 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.123 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:05.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:05.124 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-14T22:54:05.124 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 4 2024-09-14T22:54:05.124 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:05.124 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:05.125 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:05.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:05.130 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:05.130 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:05.146 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:05.146 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:05.162 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:05.163 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:05.166 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:05.165+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:05.179 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:05.179 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:05.195 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:05.195 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:05.211 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:05.211 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:05.263 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:05.262+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:05.269 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:05.269 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:05.277 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:05.364 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:05.363+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:05.419 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:05.420 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:05.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:05.471 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:05.470+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:05.531 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:05.532 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:05.540 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:05.581 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:05.580+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:05.666 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:05.665+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:05.680 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T22:54:05.688 INFO:teuthology.orchestra.run.smithi076.stdout:2870 writesame 0x1a6c836 thru 0x1a77a99 data_size 0x62(0xb264 bytes) 2024-09-14T22:54:05.692 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:05.692 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:05.701 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:05.769 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:05.767+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:05.791 INFO:teuthology.orchestra.run.smithi076.stdout:2871 write 0xb859764 thru 0xb85be60 (0x26fd bytes) 2024-09-14T22:54:05.810 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:05.810 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:05.819 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:05.854 INFO:teuthology.orchestra.run.smithi076.stdout:2872 write 0x2672a09 thru 0x267cf20 (0xa518 bytes) 2024-09-14T22:54:05.898 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:05.897+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:05.927 INFO:teuthology.orchestra.run.smithi076.stdout:2873 punch from 0xf74d59 to 0xf7eb6c, (0x9e13 bytes) 2024-09-14T22:54:05.936 INFO:teuthology.orchestra.run.smithi076.stdout:2876 write 0xe809795 thru 0xe816276 (0xcae2 bytes) 2024-09-14T22:54:05.938 INFO:teuthology.orchestra.run.smithi076.stdout:2877 punch from 0x226cc8c to 0x2270970, (0x3ce4 bytes) 2024-09-14T22:54:05.938 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:05.938 INFO:teuthology.orchestra.run.smithi076.stdout:2881 read 0x36c0189 thru 0x36c9675 (0x94ed bytes) 2024-09-14T22:54:05.938 INFO:teuthology.orchestra.run.smithi076.stdout:2882 read 0x2559101 thru 0x25600c3 (0x6fc3 bytes) 2024-09-14T22:54:05.938 INFO:teuthology.orchestra.run.smithi076.stdout:2884 write 0xd412010 thru 0xd4168d5 (0x48c6 bytes) 2024-09-14T22:54:05.960 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:05.960 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:05.969 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:05.970 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:05.969+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:06.002 INFO:teuthology.orchestra.run.smithi076.stdout:2885 read 0x43075f9 thru 0x430f0fc (0x7b04 bytes) 2024-09-14T22:54:06.002 INFO:teuthology.orchestra.run.smithi076.stdout:2886 writesame 0x943debf thru 0x944caa2 data_size 0x4(0xebe4 bytes) 2024-09-14T22:54:06.033 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:54:06.072 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:06.070+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:06.073 INFO:teuthology.orchestra.run.smithi076.stdout:2888 write 0x1c1fed5 thru 0x1c23c03 (0x3d2f bytes) 2024-09-14T22:54:06.124 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:06.124 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:06.133 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:06.166 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:06.166 INFO:teuthology.orchestra.run.smithi076.stdout:2890 read 0x1918468 thru 0x1920a60 (0x85f9 bytes) 2024-09-14T22:54:06.178 INFO:teuthology.orchestra.run.smithi076.stdout:2892 trunc from 0xe816277 to 0x4d9c881 2024-09-14T22:54:06.178 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:06.177+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:06.273 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:06.271+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:06.293 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:06.293 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:06.305 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:06.373 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:06.371+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:06.474 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:06.472+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:06.491 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:06.491 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:06.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:06.584 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:06.583+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:06.645 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:06.645 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:06.653 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:06.695 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:06.693+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:06.776 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:06.775+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:06.797 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:06.797 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:06.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:06.877 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:06.876+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:06.950 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:06.950 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:06.958 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:07.002 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:07.001+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:07.042 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:54:07.058 INFO:teuthology.orchestra.run.smithi076.stdout:2894 write 0x568b5cf thru 0x5692a5f (0x7491 bytes) 2024-09-14T22:54:07.067 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:07.067 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:07.067 INFO:teuthology.orchestra.run.smithi076.stdout:2896 write 0x7b0ac23 thru 0x7b0c26f (0x164d bytes) 2024-09-14T22:54:07.074 INFO:teuthology.orchestra.run.smithi076.stdout:2898 writesame 0x80c8326 thru 0x80ce062 data_size 0x1(0x5d3d bytes) 2024-09-14T22:54:07.075 INFO:teuthology.orchestra.run.smithi076.stdout:2899 write 0xc3aed8 thru 0xc3faa6 (0x4bcf bytes) 2024-09-14T22:54:07.076 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:07.084 INFO:teuthology.orchestra.run.smithi076.stdout:2900 trunc from 0x80ce063 to 0x259fb91 2024-09-14T22:54:07.084 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:07.083+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:07.095 INFO:teuthology.orchestra.run.smithi076.stdout:2901 read 0x921db4 thru 0x92b540 (0x978d bytes) 2024-09-14T22:54:07.108 INFO:teuthology.orchestra.run.smithi076.stdout:2902 write 0x8b767a5 thru 0x8b7c9fd (0x6259 bytes) 2024-09-14T22:54:07.111 INFO:teuthology.orchestra.run.smithi076.stdout:2903 punch from 0x263b2c4 to 0x26447f9, (0x9535 bytes) 2024-09-14T22:54:07.118 INFO:teuthology.orchestra.run.smithi076.stdout:2904 trunc from 0x8b7c9fe to 0xe7bdd33 2024-09-14T22:54:07.123 INFO:teuthology.orchestra.run.smithi076.stdout:2905 read 0xbf039d2 thru 0xbf12340 (0xe96f bytes) 2024-09-14T22:54:07.123 INFO:teuthology.orchestra.run.smithi076.stdout:2906 writesame 0x93bc5d4 thru 0x93c3837 data_size 0x2(0x7264 bytes) 2024-09-14T22:54:07.135 INFO:teuthology.orchestra.run.smithi076.stdout:2907 punch from 0xcdb0d91 to 0xcdbeee4, (0xe153 bytes) 2024-09-14T22:54:07.135 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.141 INFO:teuthology.orchestra.run.smithi076.stdout:2909 write 0xedeb4e3 thru 0xedf0a88 (0x55a6 bytes) 2024-09-14T22:54:07.144 INFO:teuthology.orchestra.run.smithi076.stdout:2910 writesame 0x39e4402 thru 0x39f28de data_size 0x1(0xe4dd bytes) 2024-09-14T22:54:07.179 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:07.178+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:07.181 INFO:teuthology.orchestra.run.smithi076.stdout:2912 writesame 0xe4b8c67 thru 0xe4be349 data_size 0x1d(0x56e3 bytes) 2024-09-14T22:54:07.183 INFO:teuthology.orchestra.run.smithi076.stdout:2913 writesame 0x26578ac thru 0x265d23d data_size 0x5(0x5992 bytes) 2024-09-14T22:54:07.187 INFO:teuthology.orchestra.run.smithi076.stdout:2915 write 0x73cab6e thru 0x73d6280 (0xb713 bytes) 2024-09-14T22:54:07.189 INFO:teuthology.orchestra.run.smithi076.stdout:2917 read 0xc1a3429 thru 0xc1a57d8 (0x23b0 bytes) 2024-09-14T22:54:07.189 INFO:teuthology.orchestra.run.smithi076.stdout:2918 writesame 0x607bcbf thru 0x608a7cf data_size 0xd(0xeb11 bytes) 2024-09-14T22:54:07.194 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.194 INFO:teuthology.orchestra.run.smithi076.stdout:2922 write 0x8bcab97 thru 0x8bd8472 (0xd8dc bytes) 2024-09-14T22:54:07.196 INFO:teuthology.orchestra.run.smithi076.stdout:2925 write 0x20d1dc1 thru 0x20e0712 (0xe952 bytes) 2024-09-14T22:54:07.199 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:07.199 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:07.209 INFO:teuthology.orchestra.run.smithi076.stdout:2928 trunc from 0xedf0a89 to 0x5caa710 2024-09-14T22:54:07.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:07.220 INFO:teuthology.orchestra.run.smithi076.stdout:2929 writesame 0x4593b45 thru 0x45a0d02 data_size 0x6(0xd1be bytes) 2024-09-14T22:54:07.222 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:07.227 INFO:teuthology.orchestra.run.smithi076.stdout:2930 punch from 0x1373d36 to 0x1382b34, (0xedfe bytes) 2024-09-14T22:54:07.236 INFO:teuthology.orchestra.run.smithi076.stdout:2931 trunc from 0x5caa710 to 0xdb557ba 2024-09-14T22:54:07.242 INFO:teuthology.orchestra.run.smithi076.stdout:2932 read 0x7cfe7b4 thru 0x7d06017 (0x7864 bytes) 2024-09-14T22:54:07.245 INFO:teuthology.orchestra.run.smithi076.stdout:2933 trunc from 0xdb557ba to 0xb7e4ee3 2024-09-14T22:54:07.252 INFO:teuthology.orchestra.run.smithi076.stdout:2934 read 0x2c1bc69 thru 0x2c1cfc9 (0x1361 bytes) 2024-09-14T22:54:07.253 INFO:teuthology.orchestra.run.smithi076.stdout:2935 write 0x694b339 thru 0x694df4e (0x2c16 bytes) 2024-09-14T22:54:07.254 INFO:teuthology.orchestra.run.smithi076.stdout:2937 punch from 0x5be624c to 0x5bf60af, (0xfe63 bytes) 2024-09-14T22:54:07.255 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.255 INFO:teuthology.orchestra.run.smithi076.stdout:2942 write 0x33c1108 thru 0x33c4b35 (0x3a2e bytes) 2024-09-14T22:54:07.256 INFO:teuthology.orchestra.run.smithi076.stdout:2943 write 0x588c9d2 thru 0x5899fab (0xd5da bytes) 2024-09-14T22:54:07.258 INFO:teuthology.orchestra.run.smithi076.stdout:2945 writesame 0x293eda thru 0x29b522 data_size 0x1(0x7649 bytes) 2024-09-14T22:54:07.280 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:07.278+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:07.282 INFO:teuthology.orchestra.run.smithi076.stdout:2946 punch from 0x70d691 to 0x71b16d, (0xdadc bytes) 2024-09-14T22:54:07.282 INFO:teuthology.orchestra.run.smithi076.stdout:2948 writesame 0x9cadce1 thru 0x9cb00d1 data_size 0x1(0x23f1 bytes) 2024-09-14T22:54:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:2949 read 0x181d19c thru 0x1827395 (0xa1fa bytes) 2024-09-14T22:54:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:2950 writesame 0x7ad7ddf thru 0x7ae4b4a data_size 0x2(0xcd6c bytes) 2024-09-14T22:54:07.309 INFO:teuthology.orchestra.run.smithi076.stdout:2955 punch from 0x86cc931 to 0x86d37bb, (0x6e8a bytes) 2024-09-14T22:54:07.309 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.309 INFO:teuthology.orchestra.run.smithi076.stdout:2959 read 0xaebfc18 thru 0xaec356c (0x3955 bytes) 2024-09-14T22:54:07.310 INFO:teuthology.orchestra.run.smithi076.stdout:2960 read 0x68a55e6 thru 0x68aa730 (0x514b bytes) 2024-09-14T22:54:07.310 INFO:teuthology.orchestra.run.smithi076.stdout:2961 read 0xa7d7813 thru 0xa7dfcb6 (0x84a4 bytes) 2024-09-14T22:54:07.311 INFO:teuthology.orchestra.run.smithi076.stdout:2962 read 0x5f15afb thru 0x5f225fb (0xcb01 bytes) 2024-09-14T22:54:07.312 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.312 INFO:teuthology.orchestra.run.smithi076.stdout:2964 punch from 0x520e2e3 to 0x520ea2a, (0x747 bytes) 2024-09-14T22:54:07.312 INFO:teuthology.orchestra.run.smithi076.stdout:2965 read 0x2e9c6b1 thru 0x2ea050d (0x3e5d bytes) 2024-09-14T22:54:07.312 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.312 INFO:teuthology.orchestra.run.smithi076.stdout:2968 write 0xa7a49c8 thru 0xa7a721a (0x2853 bytes) 2024-09-14T22:54:07.313 INFO:teuthology.orchestra.run.smithi076.stdout:2969 write 0x3623461 thru 0x362766c (0x420c bytes) 2024-09-14T22:54:07.315 INFO:teuthology.orchestra.run.smithi076.stdout:2970 write 0xac22d00 thru 0xac2494b (0x1c4c bytes) 2024-09-14T22:54:07.317 INFO:teuthology.orchestra.run.smithi076.stdout:2971 write 0x4654a31 thru 0x46641ba (0xf78a bytes) 2024-09-14T22:54:07.318 INFO:teuthology.orchestra.run.smithi076.stdout:2972 read 0x16f03ff thru 0x16f7599 (0x719b bytes) 2024-09-14T22:54:07.319 INFO:teuthology.orchestra.run.smithi076.stdout:2973 punch from 0xa017c03 to 0xa025fdf, (0xe3dc bytes) 2024-09-14T22:54:07.322 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:07.322 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:07.329 INFO:teuthology.orchestra.run.smithi076.stdout:2974 trunc from 0xb7e4ee3 to 0x36f7d94 2024-09-14T22:54:07.331 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:07.339 INFO:teuthology.orchestra.run.smithi076.stdout:2976 read 0xe8c43f thru 0xe90572 (0x4134 bytes) 2024-09-14T22:54:07.340 INFO:teuthology.orchestra.run.smithi076.stdout:2977 read 0x142b3d0 thru 0x14392ed (0xdf1e bytes) 2024-09-14T22:54:07.340 INFO:teuthology.orchestra.run.smithi076.stdout:2980 punch from 0x2ba8af1 to 0x2bb08d8, (0x7de7 bytes) 2024-09-14T22:54:07.341 INFO:teuthology.orchestra.run.smithi076.stdout:2981 read 0x290f125 thru 0x291eae9 (0xf9c5 bytes) 2024-09-14T22:54:07.351 INFO:teuthology.orchestra.run.smithi076.stdout:2982 write 0x683e29e thru 0x6849d62 (0xbac5 bytes) 2024-09-14T22:54:07.352 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.352 INFO:teuthology.orchestra.run.smithi076.stdout:2984 read 0x5ee81a8 thru 0x5ef8020 (0xfe79 bytes) 2024-09-14T22:54:07.353 INFO:teuthology.orchestra.run.smithi076.stdout:2985 write 0x6386a97 thru 0x6392983 (0xbeed bytes) 2024-09-14T22:54:07.354 INFO:teuthology.orchestra.run.smithi076.stdout:2986 read 0x3f2582f thru 0x3f2b980 (0x6152 bytes) 2024-09-14T22:54:07.363 INFO:teuthology.orchestra.run.smithi076.stdout:2987 trunc from 0x6849d63 to 0xd224577 2024-09-14T22:54:07.367 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:07.368 INFO:teuthology.orchestra.run.smithi076.stdout:2988 writesame 0x7b76ce4 thru 0x7b8020c data_size 0x7(0x9529 bytes) 2024-09-14T22:54:07.372 INFO:teuthology.orchestra.run.smithi076.stdout:2989 write 0x3b7544b thru 0x3b7fddb (0xa991 bytes) 2024-09-14T22:54:07.381 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:07.379+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:07.385 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:07.385 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:07.388 INFO:teuthology.orchestra.run.smithi076.stdout:2990 trunc from 0xd224577 to 0x10ccf91 2024-09-14T22:54:07.402 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:07.402 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:07.419 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:07.420 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:07.425 INFO:teuthology.orchestra.run.smithi076.stdout:2991 trunc from 0x10ccf91 to 0xae37942 2024-09-14T22:54:07.430 INFO:teuthology.orchestra.run.smithi076.stdout:2994 write 0x9580cc4 thru 0x9583149 (0x2486 bytes) 2024-09-14T22:54:07.431 INFO:teuthology.orchestra.run.smithi076.stdout:2995 write 0x9626efd thru 0x962b7b7 (0x48bb bytes) 2024-09-14T22:54:07.436 INFO:teuthology.orchestra.run.smithi076.stdout:2997 trunc from 0xae37942 to 0x7a8f2a4 2024-09-14T22:54:07.438 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:07.438 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:07.443 INFO:teuthology.orchestra.run.smithi076.stdout:2999 read 0x25e873a thru 0x25f5deb (0xd6b2 bytes) 2024-09-14T22:54:07.443 INFO:teuthology.orchestra.run.smithi076.stdout:3000 read 0x3752e41 thru 0x3758515 (0x56d5 bytes) 2024-09-14T22:54:07.444 INFO:teuthology.orchestra.run.smithi076.stdout:3002 punch from 0x550bfe2 to 0x5517ede, (0xbefc bytes) 2024-09-14T22:54:07.444 INFO:teuthology.orchestra.run.smithi076.stdout:3003 read 0x34c2e29 thru 0x34cd21c (0xa3f4 bytes) 2024-09-14T22:54:07.444 INFO:teuthology.orchestra.run.smithi076.stdout:3004 read 0x283e06e thru 0x2848e9b (0xae2e bytes) 2024-09-14T22:54:07.445 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:07.445 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:07.450 INFO:teuthology.orchestra.run.smithi076.stdout:3005 writesame 0x8509ed0 thru 0x850e412 data_size 0x11(0x4543 bytes) 2024-09-14T22:54:07.450 INFO:teuthology.orchestra.run.smithi076.stdout:3006 writesame 0x7d50a6e thru 0x7d5e29e data_size 0x1(0xd831 bytes) 2024-09-14T22:54:07.454 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:07.456 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:07.456 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:07.473 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:07.473 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:07.479 INFO:teuthology.orchestra.run.smithi076.stdout:3008 punch from 0x7a64e75 to 0x7a693fe, (0x4589 bytes) 2024-09-14T22:54:07.479 INFO:teuthology.orchestra.run.smithi076.stdout:3009 read 0x7f678db thru 0x7f67a58 (0x17e bytes) 2024-09-14T22:54:07.481 INFO:teuthology.orchestra.run.smithi076.stdout:3010 read 0x3cd111a thru 0x3cd980f (0x86f6 bytes) 2024-09-14T22:54:07.482 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:07.480+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:07.490 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:07.490 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:07.491 INFO:teuthology.orchestra.run.smithi076.stdout:3011 write 0xacca9e2 thru 0xacd8e39 (0xe458 bytes) 2024-09-14T22:54:07.500 INFO:teuthology.orchestra.run.smithi076.stdout:3012 trunc from 0xacd8e3a to 0x4973305 2024-09-14T22:54:07.517 INFO:teuthology.orchestra.run.smithi076.stdout:3013 write 0x7475202 thru 0x74763ca (0x11c9 bytes) 2024-09-14T22:54:07.520 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.520 INFO:teuthology.orchestra.run.smithi076.stdout:3015 write 0x45481b thru 0x463f5e (0xf744 bytes) 2024-09-14T22:54:07.522 INFO:teuthology.orchestra.run.smithi076.stdout:3017 read 0xe6c409 thru 0xe6c69f (0x297 bytes) 2024-09-14T22:54:07.522 INFO:teuthology.orchestra.run.smithi076.stdout:3018 read 0x65cc5f7 thru 0x65d8f39 (0xc943 bytes) 2024-09-14T22:54:07.522 INFO:teuthology.orchestra.run.smithi076.stdout:3020 read 0x430b9da thru 0x430e1ed (0x2814 bytes) 2024-09-14T22:54:07.523 INFO:teuthology.orchestra.run.smithi076.stdout:3021 read 0x5b2dc60 thru 0x5b2dd4b (0xec bytes) 2024-09-14T22:54:07.523 INFO:teuthology.orchestra.run.smithi076.stdout:3022 punch from 0x5979e27 to 0x5984ed8, (0xb0b1 bytes) 2024-09-14T22:54:07.524 INFO:teuthology.orchestra.run.smithi076.stdout:3024 read 0x272516d thru 0x272daa2 (0x8936 bytes) 2024-09-14T22:54:07.524 INFO:teuthology.orchestra.run.smithi076.stdout:3026 read 0x12cecaa thru 0x12d7a5c (0x8db3 bytes) 2024-09-14T22:54:07.524 INFO:teuthology.orchestra.run.smithi076.stdout:3028 read 0xb01188 thru 0xb012a4 (0x11d bytes) 2024-09-14T22:54:07.529 INFO:teuthology.orchestra.run.smithi076.stdout:3029 writesame 0x74f4db9 thru 0x7502243 data_size 0x7(0xd48b bytes) 2024-09-14T22:54:07.530 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.530 INFO:teuthology.orchestra.run.smithi076.stdout:3031 read 0x6c60569 thru 0x6c65916 (0x53ae bytes) 2024-09-14T22:54:07.530 INFO:teuthology.orchestra.run.smithi076.stdout:3032 writesame 0x1a2ec4c thru 0x1a2f77b data_size 0x2(0xb30 bytes) 2024-09-14T22:54:07.534 INFO:teuthology.orchestra.run.smithi076.stdout:3033 read 0x7015933 thru 0x7020073 (0xa741 bytes) 2024-09-14T22:54:07.534 INFO:teuthology.orchestra.run.smithi076.stdout:3034 write 0x692921e thru 0x692d6e6 (0x44c9 bytes) 2024-09-14T22:54:07.535 INFO:teuthology.orchestra.run.smithi076.stdout:3035 read 0x342f236 thru 0x343a32d (0xb0f8 bytes) 2024-09-14T22:54:07.536 INFO:teuthology.orchestra.run.smithi076.stdout:3036 punch from 0x1b5f4b0 to 0x1b6e701, (0xf251 bytes) 2024-09-14T22:54:07.536 INFO:teuthology.orchestra.run.smithi076.stdout:3037 trunc from 0x7502244 to 0x6f1b7ae 2024-09-14T22:54:07.544 INFO:teuthology.orchestra.run.smithi076.stdout:3038 write 0x24c0546 thru 0x24c5d1b (0x57d6 bytes) 2024-09-14T22:54:07.547 INFO:teuthology.orchestra.run.smithi076.stdout:3040 read 0x2e8408e thru 0x2e8fcd6 (0xbc49 bytes) 2024-09-14T22:54:07.547 INFO:teuthology.orchestra.run.smithi076.stdout:3041 writesame 0x1eeb672 thru 0x1ef6063 data_size 0x9(0xa9f2 bytes) 2024-09-14T22:54:07.551 INFO:teuthology.orchestra.run.smithi076.stdout:3043 write 0x27e6347 thru 0x27e88b0 (0x256a bytes) 2024-09-14T22:54:07.569 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116113 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5611520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.512954973999999, 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053031104000000003, 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116113 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053879+0000", 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053986+0000", 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010658 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053989+0000", 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3579999999999999e-06 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.053998+0000", 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8319999999999995e-06 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054004+0000", 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3579999999999999e-06 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106732+0000", 2024-09-14T22:54:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052727505000000001 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106816+0000", 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4482999999999997e-05 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106910+0000", 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3987999999999995e-05 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.512819275999998, 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:54:07.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.512624184, 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:54:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:54:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.512471393999999, 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:54:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:54:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.512266371999999, 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:54:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:54:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.511995706, 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:54:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:54:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.511733545999999, 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.511574694, 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:54:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:54:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.511387897999999, 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.511185652999998, 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:54:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.510977295, 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:54:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:54:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.510663577999999, 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.510457815999999, 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:54:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:54:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:54:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:54:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.510288939999999, 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:54:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.510018246000001, 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:54:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.509868971, 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:54:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 31.509627729000002, 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.200937697000001, 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:54:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:54:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153281 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'291, mlcod=72'289)", 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.1314282349999996, 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060467871999999999, 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435411+0000", 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5589999999999999e-06 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435419+0000", 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.909e-06 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435423+0000", 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5140000000000003e-06 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435458+0000", 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965000000000002e-05 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435470+0000", 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1559999999999999e-05 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495827+0000", 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060356819999999999 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495874+0000", 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7003999999999999e-05 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153282 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'292, mlcod=72'289)", 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.1307965190000004, 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060246112999999997, 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436038+0000", 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436042+0000", 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8920000000000002e-06 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495165+0000", 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059122945000000003 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495179+0000", 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4185000000000001e-05 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495346+0000", 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016677400000000001 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495373+0000", 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7172e-05 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496245+0000", 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00087206100000000004 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496284+0000", 2024-09-14T22:54:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8525000000000001e-05 2024-09-14T22:54:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:07.660 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:07.662 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:07.662 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:07.660+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:07.662 INFO:teuthology.orchestra.run.smithi076.stdout:3046 write 0xee0e062 thru 0xee13b3d (0x5adc bytes) 2024-09-14T22:54:07.662 INFO:teuthology.orchestra.run.smithi076.stdout:3047 punch from 0x815972c to 0x8166265, (0xcb39 bytes) 2024-09-14T22:54:07.663 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.663 INFO:teuthology.orchestra.run.smithi076.stdout:3050 read 0x5129d92 thru 0x5138422 (0xe691 bytes) 2024-09-14T22:54:07.663 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.663 INFO:teuthology.orchestra.run.smithi076.stdout:3053 punch from 0x1eb312a to 0x1ec0671, (0xd547 bytes) 2024-09-14T22:54:07.663 INFO:teuthology.orchestra.run.smithi076.stdout:3054 read 0x3eba084 thru 0x3ec2fbe (0x8f3b bytes) 2024-09-14T22:54:07.663 INFO:teuthology.orchestra.run.smithi076.stdout:3055 read 0x71ec98 thru 0x7251f5 (0x655e bytes) 2024-09-14T22:54:07.663 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:07.663 INFO:teuthology.orchestra.run.smithi076.stdout:3057 clone 29 order 20 su 131072 sc 2 2024-09-14T22:54:07.762 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:07.761+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:07.807 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:07.817 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:07.863 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:07.862+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:07.959 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:07.960 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:07.964 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:07.963+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:07.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:08.035 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:08.035 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:54:08.036 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-14T22:54:08.036 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T22:54:08.036 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T22:54:08.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-14T22:54:08.064 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:08.063+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:08.076 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.178739391000001, 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:54:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:54:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127349 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4120576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.553049834999999, 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053205084, 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127349 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7000000000000005e-07 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521644+0000", 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521715+0000", 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0736999999999997e-05 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521725+0000", 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0237999999999999e-05 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574705+0000", 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052979928000000003 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574768+0000", 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2618000000000003e-05 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574846+0000", 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8143e-05 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.552945271999999, 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:54:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.552857280000001, 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.552623362999999, 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:54:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:54:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.552442467999999, 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.552142965000002, 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:54:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127361 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.551822819000002, 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053260788000000003, 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127361 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522869+0000", 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522872+0000", 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.066e-06 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522879+0000", 2024-09-14T22:54:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8110000000000003e-06 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522885+0000", 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.003e-06 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575959+0000", 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073515000000002 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576037+0000", 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7991999999999995e-05 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576129+0000", 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1772000000000002e-05 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.551796755000002, 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:54:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:54:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.551433027000002, 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:54:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.550843409999999, 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:54:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:54:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.550641172999999, 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:54:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:54:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.550451362, 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.550139358999999, 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:54:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:54:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.549896872000001, 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:54:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.549737910000001, 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.549103639999998, 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:54:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.54888953, 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:54:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:54:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.548753059999999, 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:54:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.548500579999999, 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:54:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.166 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:08.168 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone28 from 0xee13b3e (overlap 0x10ccf91) to 0xf1ee02 2024-09-14T22:54:08.169 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:08.169 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:08.167+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:08.170 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:08.170 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T22:54:08.170 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:08.265 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T22:54:08.268 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:08.267+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:08.315 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:08.315 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:08.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:08.345 INFO:teuthology.orchestra.run.smithi076.stdout:3058 trunc from 0xee13b3e to 0xdb63b85 2024-09-14T22:54:08.360 INFO:teuthology.orchestra.run.smithi076.stdout:3059 write 0x488d3d8 thru 0x489ce30 (0xfa59 bytes) 2024-09-14T22:54:08.365 INFO:teuthology.orchestra.run.smithi076.stdout:3060 punch from 0x1a47d5b to 0x1a555c7, (0xd86c bytes) 2024-09-14T22:54:08.365 INFO:teuthology.orchestra.run.smithi076.stdout:3061 read 0x81050cd thru 0x8113219 (0xe14d bytes) 2024-09-14T22:54:08.365 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:08.369 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:08.368+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:08.369 INFO:teuthology.orchestra.run.smithi076.stdout:3064 trunc from 0xdb63b85 to 0x9d62125 2024-09-14T22:54:08.385 INFO:teuthology.orchestra.run.smithi076.stdout:3066 punch from 0x7e2163e to 0x7e26cad, (0x566f bytes) 2024-09-14T22:54:08.394 INFO:teuthology.orchestra.run.smithi076.stdout:3068 writesame 0xc91cf8f thru 0xc92bd32 data_size 0x43(0xeda4 bytes) 2024-09-14T22:54:08.400 INFO:teuthology.orchestra.run.smithi076.stdout:3069 write 0xde80260 thru 0xde8b546 (0xb2e7 bytes) 2024-09-14T22:54:08.403 INFO:teuthology.orchestra.run.smithi076.stdout:3070 punch from 0x5301842 to 0x5308282, (0x6a40 bytes) 2024-09-14T22:54:08.411 INFO:teuthology.orchestra.run.smithi076.stdout:3072 write 0xecdc6b2 thru 0xece9b95 (0xd4e4 bytes) 2024-09-14T22:54:08.415 INFO:teuthology.orchestra.run.smithi076.stdout:3073 punch from 0xd1389fc to 0xd13b081, (0x2685 bytes) 2024-09-14T22:54:08.415 INFO:teuthology.orchestra.run.smithi076.stdout:3075 write 0x8ce4425 thru 0x8ceb65e (0x723a bytes) 2024-09-14T22:54:08.419 INFO:teuthology.orchestra.run.smithi076.stdout:3076 read 0xa847961 thru 0xa8537c7 (0xbe67 bytes) 2024-09-14T22:54:08.419 INFO:teuthology.orchestra.run.smithi076.stdout:3077 read 0xbd972b5 thru 0xbda3e6b (0xcbb7 bytes) 2024-09-14T22:54:08.419 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:08.419 INFO:teuthology.orchestra.run.smithi076.stdout:3081 punch from 0x18439ea to 0x184618e, (0x27a4 bytes) 2024-09-14T22:54:08.419 INFO:teuthology.orchestra.run.smithi076.stdout:3082 writesame 0xe903d80 thru 0xe909761 data_size 0x4e(0x59e2 bytes) 2024-09-14T22:54:08.423 INFO:teuthology.orchestra.run.smithi076.stdout:3084 write 0xbbd938e thru 0xbbdf2f6 (0x5f69 bytes) 2024-09-14T22:54:08.428 INFO:teuthology.orchestra.run.smithi076.stdout:3085 punch from 0x5cea1aa to 0x5ceaf93, (0xde9 bytes) 2024-09-14T22:54:08.428 INFO:teuthology.orchestra.run.smithi076.stdout:3086 punch from 0xdddb181 to 0xdde5380, (0xa1ff bytes) 2024-09-14T22:54:08.428 INFO:teuthology.orchestra.run.smithi076.stdout:3088 read 0xa9df149 thru 0xa9eb797 (0xc64f bytes) 2024-09-14T22:54:08.428 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:08.429 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:08.429 INFO:teuthology.orchestra.run.smithi076.stdout:3091 writesame 0x728ca28 thru 0x72948db data_size 0x44(0x7eb4 bytes) 2024-09-14T22:54:08.431 INFO:teuthology.orchestra.run.smithi076.stdout:3092 write 0xb6270e2 thru 0xb6359dc (0xe8fb bytes) 2024-09-14T22:54:08.435 INFO:teuthology.orchestra.run.smithi076.stdout:3093 read 0x1933536 thru 0x193beb8 (0x8983 bytes) 2024-09-14T22:54:08.436 INFO:teuthology.orchestra.run.smithi076.stdout:3094 read 0xc1885ab thru 0xc19001b (0x7a71 bytes) 2024-09-14T22:54:08.436 INFO:teuthology.orchestra.run.smithi076.stdout:3095 write 0x443c14f thru 0x44493dd (0xd28f bytes) 2024-09-14T22:54:08.443 INFO:teuthology.orchestra.run.smithi076.stdout:3096 write 0x99fc84c thru 0x99fe934 (0x20e9 bytes) 2024-09-14T22:54:08.448 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:08.449 INFO:teuthology.orchestra.run.smithi076.stdout:3098 read 0xd54e8fb thru 0xd553be9 (0x52ef bytes) 2024-09-14T22:54:08.449 INFO:teuthology.orchestra.run.smithi076.stdout:3101 write 0x38efb94 thru 0x38f5342 (0x57af bytes) 2024-09-14T22:54:08.454 INFO:teuthology.orchestra.run.smithi076.stdout:3103 punch from 0x2e0bca7 to 0x2e14c7c, (0x8fd5 bytes) 2024-09-14T22:54:08.454 INFO:teuthology.orchestra.run.smithi076.stdout:3104 write 0x40e7d51 thru 0x40e9bbf (0x1e6f bytes) 2024-09-14T22:54:08.459 INFO:teuthology.orchestra.run.smithi076.stdout:3105 read 0x8af83c3 thru 0x8b001fa (0x7e38 bytes) 2024-09-14T22:54:08.459 INFO:teuthology.orchestra.run.smithi076.stdout:3107 writesame 0x64c8ca7 thru 0x64ce56b data_size 0xf(0x58c5 bytes) 2024-09-14T22:54:08.464 INFO:teuthology.orchestra.run.smithi076.stdout:3108 write 0x123f3ac thru 0x1241c08 (0x285d bytes) 2024-09-14T22:54:08.470 INFO:teuthology.orchestra.run.smithi076.stdout:3109 flatten 2024-09-14T22:54:08.470 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:08.469+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:08.479 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:08.480 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:08.494 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:08.536 INFO:teuthology.orchestra.run.smithi076.stdout:3110 writesame 0xad8f64b thru 0xad91845 data_size 0x1(0x21fb bytes) 2024-09-14T22:54:08.557 INFO:teuthology.orchestra.run.smithi076.stdout:3114 trunc from 0xece9b96 to 0x2927489 2024-09-14T22:54:08.570 INFO:teuthology.orchestra.run.smithi076.stdout:3116 punch from 0x2498b9 to 0x2525d2, (0x8d19 bytes) 2024-09-14T22:54:08.571 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:08.569+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:08.579 INFO:teuthology.orchestra.run.smithi076.stdout:3117 trunc from 0x2927489 to 0xa410b37 2024-09-14T22:54:08.585 INFO:teuthology.orchestra.run.smithi076.stdout:3118 clone 30 order 24 su 262144 sc 2 2024-09-14T22:54:08.612 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.529000633000003, 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.528782737, 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:54:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.526099090999999, 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:54:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:54:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:54:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.525940675999998, 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:54:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.525913410000001, 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.525706268999997, 2024-09-14T22:54:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:54:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:54:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.525672004999997, 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.524039959, 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:54:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.524015347000002, 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.523881060999997, 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:54:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:54:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:54:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.522831353000001, 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:54:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.522729167999998, 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:54:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:54:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.522705264000003, 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.522671744, 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:54:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.522517098000002, 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.522475923999998, 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:54:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.522447763999999, 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.522258373, 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:54:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.606168545999999, 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:54:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.606148715000003, 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:54:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:54:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:08.707 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:08.709 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:08.709 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:08.707+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:08.809 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:08.808+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:08.853 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:08.853 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:08.853 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:08.853 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:08.861 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:08.910 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:08.909+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:08.964 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:08.964 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:08.964 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:08.973 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:09.011 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:09.010+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:09.057 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:54:09.112 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:09.111+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:09.123 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:09.123 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:09.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:09.123 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:09.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.124 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:54:09.124 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:09.124 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.613309454000003, 2024-09-14T22:54:09.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:54:09.124 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:54:09.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:54:09.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:09.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:54:09.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:54:09.131 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.131 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.131 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.748651598999999, 2024-09-14T22:54:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:54:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:54:09.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:54:09.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:54:09.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:54:09.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:54:09.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:54:09.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.137 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.748545663000002, 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.138 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:54:09.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:09.143 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.748470961999999, 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:54:09.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.748385481, 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:54:09.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.152 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.748239095000002, 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:54:09.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.156 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.748189095000001, 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:54:09.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:54:09.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:09.161 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.748025931000001, 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:54:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:54:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.747987621, 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.747853952, 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:54:09.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.747773385000002, 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.747676448, 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.747624893000001, 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:54:09.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:54:09.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.744500264000003, 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.188 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:54:09.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.744368651000002, 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:54:09.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.744082239000001, 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:54:09.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.201 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.6860459849999998, 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:54:09.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:54:09.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:54:09.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.6860061569999996, 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:54:09.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:54:09.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.6859901419999996, 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:09.214 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:09.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:54:09.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:54:09.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:09.220 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.636861787, 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:09.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:54:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:54:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:09.226 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:09.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:09.229 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:09.226+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:09.262 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T22:54:09.285 INFO:teuthology.orchestra.run.smithi076.stdout:3119 punch from 0x3747020 to 0x3748a23, (0x1a03 bytes) 2024-09-14T22:54:09.291 INFO:teuthology.orchestra.run.smithi076.stdout:3120 trunc from 0xa410b37 to 0xacd2578 2024-09-14T22:54:09.295 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:09.304 INFO:teuthology.orchestra.run.smithi076.stdout:3122 write 0xd8e2bcc thru 0xd8eb7a5 (0x8bda bytes) 2024-09-14T22:54:09.308 INFO:teuthology.orchestra.run.smithi076.stdout:3123 read 0x6ce232a thru 0x6ce4294 (0x1f6b bytes) 2024-09-14T22:54:09.308 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:09.308 INFO:teuthology.orchestra.run.smithi076.stdout:3127 write 0x4e3c527 thru 0x4e4a136 (0xdc10 bytes) 2024-09-14T22:54:09.318 INFO:teuthology.orchestra.run.smithi076.stdout:3128 writesame 0xe342f46 thru 0xe3445ca data_size 0x1(0x1685 bytes) 2024-09-14T22:54:09.318 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:09.318 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:09.319 INFO:teuthology.orchestra.run.smithi076.stdout:3132 write 0x5c41953 thru 0x5c4d895 (0xbf43 bytes) 2024-09-14T22:54:09.320 INFO:teuthology.orchestra.run.smithi076.stdout:3133 read 0x6a517c4 thru 0x6a55253 (0x3a90 bytes) 2024-09-14T22:54:09.320 INFO:teuthology.orchestra.run.smithi076.stdout:3134 read 0xadfba5c thru 0xadfcea4 (0x1449 bytes) 2024-09-14T22:54:09.320 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:09.327 INFO:teuthology.orchestra.run.smithi076.stdout:3136 trunc from 0xe3445cb to 0x88b0574 2024-09-14T22:54:09.329 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:09.328+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:09.340 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:09.340 INFO:teuthology.orchestra.run.smithi076.stdout:3138 write 0x47ac432 thru 0x47b1642 (0x5211 bytes) 2024-09-14T22:54:09.345 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:09.346 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:09.346 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:09.346 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:09.355 INFO:teuthology.orchestra.run.smithi076.stdout:3140 writesame 0xea1f862 thru 0xea25d35 data_size 0x1(0x64d4 bytes) 2024-09-14T22:54:09.355 INFO:teuthology.orchestra.run.smithi076.stdout:3141 writesame 0x81669da thru 0x8168d1b data_size 0x1(0x2342 bytes) 2024-09-14T22:54:09.357 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:09.362 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:09.362 INFO:teuthology.orchestra.run.smithi076.stdout:3144 punch from 0x3045c92 to 0x305273a, (0xcaa8 bytes) 2024-09-14T22:54:09.362 INFO:teuthology.orchestra.run.smithi076.stdout:3145 writesame 0x1bdb780 thru 0x1be40ab data_size 0x2(0x892c bytes) 2024-09-14T22:54:09.378 INFO:teuthology.orchestra.run.smithi076.stdout:3146 trunc from 0xea25d36 to 0xe43aa3a 2024-09-14T22:54:09.385 INFO:teuthology.orchestra.run.smithi076.stdout:3147 punch from 0x236cc07 to 0x23758fd, (0x8cf6 bytes) 2024-09-14T22:54:09.385 INFO:teuthology.orchestra.run.smithi076.stdout:3148 write 0xfa6653 thru 0xfb2019 (0xb9c7 bytes) 2024-09-14T22:54:09.393 INFO:teuthology.orchestra.run.smithi076.stdout:3149 punch from 0x4b72637 to 0x4b8201d, (0xf9e6 bytes) 2024-09-14T22:54:09.393 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:09.402 INFO:teuthology.orchestra.run.smithi076.stdout:3151 trunc from 0xe43aa3a to 0x6647f15 2024-09-14T22:54:09.423 INFO:teuthology.orchestra.run.smithi076.stdout:3152 write 0x6f7781b thru 0x6f7c404 (0x4bea bytes) 2024-09-14T22:54:09.430 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:09.428+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:09.430 INFO:teuthology.orchestra.run.smithi076.stdout:3156 trunc from 0x6f7c405 to 0xbc06ef2 2024-09-14T22:54:09.436 INFO:teuthology.orchestra.run.smithi076.stdout:3157 write 0x92bc075 thru 0x92caa76 (0xea02 bytes) 2024-09-14T22:54:09.440 INFO:teuthology.orchestra.run.smithi076.stdout:3158 write 0xa50c4d thru 0xa5c429 (0xb7dd bytes) 2024-09-14T22:54:09.441 INFO:teuthology.orchestra.run.smithi076.stdout:3160 read 0xb0ea44d thru 0xb0eeaaf (0x4663 bytes) 2024-09-14T22:54:09.441 INFO:teuthology.orchestra.run.smithi076.stdout:3161 read 0x12f6f24 thru 0x1305ed8 (0xefb5 bytes) 2024-09-14T22:54:09.441 INFO:teuthology.orchestra.run.smithi076.stdout:3162 punch from 0x95f84dd to 0x95ff292, (0x6db5 bytes) 2024-09-14T22:54:09.442 INFO:teuthology.orchestra.run.smithi076.stdout:3163 read 0x7cbc89b thru 0x7cc5a98 (0x91fe bytes) 2024-09-14T22:54:09.442 INFO:teuthology.orchestra.run.smithi076.stdout:3164 write 0x7b3b6ca thru 0x7b4a02e (0xe965 bytes) 2024-09-14T22:54:09.443 INFO:teuthology.orchestra.run.smithi076.stdout:3165 read 0x621c570 thru 0x621d736 (0x11c7 bytes) 2024-09-14T22:54:09.455 INFO:teuthology.orchestra.run.smithi076.stdout:3168 trunc from 0xbc06ef2 to 0x214fd71 2024-09-14T22:54:09.473 INFO:teuthology.orchestra.run.smithi076.stdout:3169 trunc from 0x214fd71 to 0x3214cff 2024-09-14T22:54:09.662 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:09.663 INFO:teuthology.orchestra.run.smithi076.stdout:3170 punch from 0x88585c to 0x88dadc, (0x8280 bytes) 2024-09-14T22:54:09.663 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:09.663 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:09.663 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:09.664 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:09.664 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:09.664 INFO:teuthology.orchestra.run.smithi076.stdout:3171 trunc from 0x3214cff to 0x9a41dd8 2024-09-14T22:54:09.665 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:09.666 INFO:teuthology.orchestra.run.smithi076.stdout:3172 write 0xbe4a75f thru 0xbe53049 (0x88eb bytes) 2024-09-14T22:54:09.666 INFO:teuthology.orchestra.run.smithi076.stdout:3173 punch from 0x13e4f80 to 0x13ef306, (0xa386 bytes) 2024-09-14T22:54:09.666 INFO:teuthology.orchestra.run.smithi076.stdout:3174 write 0xb748304 thru 0xb74d77d (0x547a bytes) 2024-09-14T22:54:09.667 INFO:teuthology.orchestra.run.smithi076.stdout:3176 write 0xe53ddb thru 0xe5d393 (0x95b9 bytes) 2024-09-14T22:54:09.667 INFO:teuthology.orchestra.run.smithi076.stdout:3177 writesame 0xa63e204 thru 0xa644bef data_size 0x2(0x69ec bytes) 2024-09-14T22:54:09.667 INFO:teuthology.orchestra.run.smithi076.stdout:3178 read 0x31563ed thru 0x31607ed (0xa401 bytes) 2024-09-14T22:54:09.667 INFO:teuthology.orchestra.run.smithi076.stdout:3179 write 0x514e7a2 thru 0x51564d9 (0x7d38 bytes) 2024-09-14T22:54:09.667 INFO:teuthology.orchestra.run.smithi076.stdout:3180 write 0xa43511 thru 0xa47acd (0x45bd bytes) 2024-09-14T22:54:09.667 INFO:teuthology.orchestra.run.smithi076.stdout:3181 write 0xe35b681 thru 0xe366b5d (0xb4dd bytes) 2024-09-14T22:54:09.667 INFO:teuthology.orchestra.run.smithi076.stdout:3182 clone 31 order 18 su 8192 sc 7 2024-09-14T22:54:09.667 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:09.662+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:09.764 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:09.763+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:09.814 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:09.814 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:09.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:09.814 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:09.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.814 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.814 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.962275488000003, 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:09.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.818 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.961337897999996, 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.819 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:54:09.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:54:09.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.961246535000001, 2024-09-14T22:54:09.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.827 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.960601650000001, 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:54:09.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:54:09.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:54:09.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:09.832 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.960114271000002, 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:09.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.836 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.959226995999998, 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.837 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:54:09.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:09.841 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.958634572000001, 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:09.846 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.957901755999998, 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.957802606999998, 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.851 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:54:09.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:54:09.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.855 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.957484149000003, 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.856 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:54:09.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.860 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.956974655000003, 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.861 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:09.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:54:09.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:54:09.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.956916796000002, 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:09.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.869 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.956124172999999, 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:54:09.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:54:09.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:09.874 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.955448734000001, 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:09.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.955315423999998, 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:54:09.879 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:09.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:54:09.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.954698501000003, 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.884 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.888 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.954631644000003, 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:09.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:54:09.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.954180833000002, 2024-09-14T22:54:09.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:54:09.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.954060599999998, 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.898 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:54:09.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.902 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.952742473000001, 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:09.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:54:09.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:09.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:09.907 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:09.909 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:09.909 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:09.910 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:09.910 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:09.910 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:09.911 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:09.911 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:09.911 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:09.911 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:09.912 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:09.912 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:09.912 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:09.907+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:09.916 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:09.916 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:09.933 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:09.933 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:09.951 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:09.951 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:10.009 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:10.008+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:10.021 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:10.021 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:10.030 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:10.053 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:54:10.112 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:10.111+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:10.121 INFO:teuthology.orchestra.run.smithi076.stdout:leaving image image_client.0-clone30 intact 2024-09-14T22:54:10.167 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:10.167 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:10.177 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:10.218 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:10.217+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:10.283 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T22:54:10.312 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:10.310+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:10.336 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:10.336 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:10.346 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:10.412 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:10.411+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:10.456 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:10.456 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:10.466 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:10.501 INFO:teuthology.orchestra.run.smithi076.stdout:3184 write 0xb34103a thru 0xb3415d3 (0x59a bytes) 2024-09-14T22:54:10.512 INFO:teuthology.orchestra.run.smithi076.stdout:3185 punch from 0x11521f0 to 0x115860d, (0x641d bytes) 2024-09-14T22:54:10.512 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:10.513 INFO:teuthology.orchestra.run.smithi076.stdout:3187 trunc from 0xe366b5e to 0xea5da61 2024-09-14T22:54:10.513 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:10.511+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:10.519 INFO:teuthology.orchestra.run.smithi076.stdout:3189 write 0x266d345 thru 0x2672724 (0x53e0 bytes) 2024-09-14T22:54:10.533 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:10.533 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:10.534 INFO:teuthology.orchestra.run.smithi076.stdout:3194 punch from 0x9ff13cf to 0x9ff8d56, (0x7987 bytes) 2024-09-14T22:54:10.534 INFO:teuthology.orchestra.run.smithi076.stdout:3196 read 0xc2a521d thru 0xc2b2436 (0xd21a bytes) 2024-09-14T22:54:10.534 INFO:teuthology.orchestra.run.smithi076.stdout:3199 clone 32 order 20 su 32768 sc 7 2024-09-14T22:54:10.576 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:10.577 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:10.585 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:10.614 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:10.612+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:10.714 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:10.713+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:10.727 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:10.727 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:10.736 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:10.820 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:10.819+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:10.881 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:10.881 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:10.889 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:10.930 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:10.929+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:11.018 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:11.016+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:11.031 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:11.032 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:11.040 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:11.126 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:11.125+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:11.143 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone31 from 0xea5da61 (overlap 0xe366b5e) to 0xcca93d4 2024-09-14T22:54:11.208 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:11.208 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:11.219 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:11.220 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:11.218+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:11.320 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:11.319+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:11.325 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:11.325 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:11.334 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:11.391 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T22:54:11.421 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:11.420+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:11.439 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:11.439 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:11.449 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:11.532 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:11.531+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:11.591 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:11.591 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:11.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:11.623 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:11.622+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:11.668 INFO:teuthology.orchestra.run.smithi076.stdout:3200 read 0x214d120 thru 0x2157e25 (0xad06 bytes) 2024-09-14T22:54:11.670 INFO:teuthology.orchestra.run.smithi076.stdout:3201 read 0x836d820 thru 0x8377242 (0x9a23 bytes) 2024-09-14T22:54:11.670 INFO:teuthology.orchestra.run.smithi076.stdout:3203 read 0x3b9ce8c thru 0x3b9f871 (0x29e6 bytes) 2024-09-14T22:54:11.671 INFO:teuthology.orchestra.run.smithi076.stdout:3204 writesame 0x45d8845 thru 0x45de867 data_size 0x1(0x6023 bytes) 2024-09-14T22:54:11.697 INFO:teuthology.orchestra.run.smithi076.stdout:3205 writesame 0xcf0f199 thru 0xcf0f37b data_size 0x3(0x1e3 bytes) 2024-09-14T22:54:11.701 INFO:teuthology.orchestra.run.smithi076.stdout:3206 writesame 0xb4fee73 thru 0xb50d17c data_size 0x9(0xe30a bytes) 2024-09-14T22:54:11.723 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:11.722+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:11.738 INFO:teuthology.orchestra.run.smithi076.stdout:3207 punch from 0x47a26b to 0x488940, (0xe6d5 bytes) 2024-09-14T22:54:11.738 INFO:teuthology.orchestra.run.smithi076.stdout:3208 read 0x46297bc thru 0x4632861 (0x90a6 bytes) 2024-09-14T22:54:11.746 INFO:teuthology.orchestra.run.smithi076.stdout:3209 trunc from 0xea5da61 to 0x8277580 2024-09-14T22:54:11.758 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:11.758 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:11.768 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:11.777 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:11.777 INFO:teuthology.orchestra.run.smithi076.stdout:3213 writesame 0x473f0ae thru 0x4742cbd data_size 0x3e(0x3c10 bytes) 2024-09-14T22:54:11.797 INFO:teuthology.orchestra.run.smithi076.stdout:3214 read 0x2eb70ef thru 0x2ec02e0 (0x91f2 bytes) 2024-09-14T22:54:11.798 INFO:teuthology.orchestra.run.smithi076.stdout:3215 writesame 0x6e154e5 thru 0x6e1c7c6 data_size 0x55(0x72e2 bytes) 2024-09-14T22:54:11.813 INFO:teuthology.orchestra.run.smithi076.stdout:3216 trunc from 0x8277580 to 0x163544e 2024-09-14T22:54:11.824 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:11.821+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:11.922 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:11.923 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:11.924 INFO:teuthology.orchestra.run.smithi076.stdout:3217 writesame 0xe92421b thru 0xe928c75 data_size 0x51(0x4a5b bytes) 2024-09-14T22:54:11.925 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:11.923+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:11.928 INFO:teuthology.orchestra.run.smithi076.stdout:3218 trunc from 0xe928c76 to 0xae5e199 2024-09-14T22:54:11.932 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:11.935 INFO:teuthology.orchestra.run.smithi076.stdout:3219 punch from 0x39aee92 to 0x39bbd11, (0xce7f bytes) 2024-09-14T22:54:11.936 INFO:teuthology.orchestra.run.smithi076.stdout:3220 punch from 0x1e6edda to 0x1e78369, (0x958f bytes) 2024-09-14T22:54:11.936 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:11.936 INFO:teuthology.orchestra.run.smithi076.stdout:3222 read 0x4f249cf thru 0x4f2c0dc (0x770e bytes) 2024-09-14T22:54:11.936 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:11.936 INFO:teuthology.orchestra.run.smithi076.stdout:3227 read 0x5ca7314 thru 0x5cb4ae3 (0xd7d0 bytes) 2024-09-14T22:54:11.942 INFO:teuthology.orchestra.run.smithi076.stdout:3228 write 0xc5ec116 thru 0xc5f24f3 (0x63de bytes) 2024-09-14T22:54:11.947 INFO:teuthology.orchestra.run.smithi076.stdout:3229 read 0xbda55a3 thru 0xbdaa568 (0x4fc6 bytes) 2024-09-14T22:54:11.947 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:11.948 INFO:teuthology.orchestra.run.smithi076.stdout:3231 trunc from 0xc5f24f4 to 0xbe5c975 2024-09-14T22:54:11.958 INFO:teuthology.orchestra.run.smithi076.stdout:3233 punch from 0x4d821df to 0x4d8f1fd, (0xd01e bytes) 2024-09-14T22:54:11.959 INFO:teuthology.orchestra.run.smithi076.stdout:3234 trunc from 0xbe5c975 to 0xacb4766 2024-09-14T22:54:11.962 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:11.965 INFO:teuthology.orchestra.run.smithi076.stdout:3235 write 0x461b88b thru 0x46260f9 (0xa86f bytes) 2024-09-14T22:54:11.971 INFO:teuthology.orchestra.run.smithi076.stdout:3236 punch from 0xa5611ca to 0xa56456d, (0x33a3 bytes) 2024-09-14T22:54:11.971 INFO:teuthology.orchestra.run.smithi076.stdout:3237 write 0x39942a thru 0x3a32fd (0x9ed4 bytes) 2024-09-14T22:54:11.998 INFO:teuthology.orchestra.run.smithi076.stdout:3239 punch from 0x2eb85c to 0x2facdb, (0xf47f bytes) 2024-09-14T22:54:12.005 INFO:teuthology.orchestra.run.smithi076.stdout:3240 write 0xb97d9ea thru 0xb981617 (0x3c2e bytes) 2024-09-14T22:54:12.008 INFO:teuthology.orchestra.run.smithi076.stdout:3244 writesame 0xa390e87 thru 0xa39aaf6 data_size 0x8(0x9c70 bytes) 2024-09-14T22:54:12.014 INFO:teuthology.orchestra.run.smithi076.stdout:3245 read 0x7983762 thru 0x7989470 (0x5d0f bytes) 2024-09-14T22:54:12.014 INFO:teuthology.orchestra.run.smithi076.stdout:3246 writesame 0xa458125 thru 0xa45f3fb data_size 0x1(0x72d7 bytes) 2024-09-14T22:54:12.025 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:12.023+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:12.030 INFO:teuthology.orchestra.run.smithi076.stdout:3247 read 0xa8de15d thru 0xa8dec2d (0xad1 bytes) 2024-09-14T22:54:12.040 INFO:teuthology.orchestra.run.smithi076.stdout:3248 trunc from 0xb981618 to 0x44b2104 2024-09-14T22:54:12.052 INFO:teuthology.orchestra.run.smithi076.stdout:3249 punch from 0x15f417c to 0x1602389, (0xe20d bytes) 2024-09-14T22:54:12.052 INFO:teuthology.orchestra.run.smithi076.stdout:3251 read 0x326c3c8 thru 0x32792ff (0xcf38 bytes) 2024-09-14T22:54:12.052 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.053 INFO:teuthology.orchestra.run.smithi076.stdout:3254 read 0x39e7cda thru 0x39f0893 (0x8bba bytes) 2024-09-14T22:54:12.063 INFO:teuthology.orchestra.run.smithi076.stdout:3256 write 0x83ada1e thru 0x83b202a (0x460d bytes) 2024-09-14T22:54:12.066 INFO:teuthology.orchestra.run.smithi076.stdout:3257 read 0x8b7251 thru 0x8bcc58 (0x5a08 bytes) 2024-09-14T22:54:12.071 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:54:12.072 INFO:teuthology.orchestra.run.smithi076.stdout:3258 trunc from 0x83b202b to 0x4147732 2024-09-14T22:54:12.084 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:12.085 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:12.094 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:12.094 INFO:teuthology.orchestra.run.smithi076.stdout:3259 write 0x9a5feba thru 0x9a647e9 (0x4930 bytes) 2024-09-14T22:54:12.099 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:12.099 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:12.109 INFO:teuthology.orchestra.run.smithi076.stdout:3261 trunc from 0x9a647ea to 0x65a2a7 2024-09-14T22:54:12.121 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:12.121 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:12.126 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:12.124+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:12.146 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:12.146 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:12.190 INFO:teuthology.orchestra.run.smithi076.stdout:3262 trunc from 0x65a2a7 to 0x1170168 2024-09-14T22:54:12.210 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:12.210 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:12.216 INFO:teuthology.orchestra.run.smithi076.stdout:3263 write 0xdeada7c thru 0xdeb282e (0x4db3 bytes) 2024-09-14T22:54:12.219 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:12.220 INFO:teuthology.orchestra.run.smithi076.stdout:3264 read 0x966dc64 thru 0x9674e71 (0x720e bytes) 2024-09-14T22:54:12.220 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.235 INFO:teuthology.orchestra.run.smithi076.stdout:3267 trunc from 0xdeb282f to 0xa0af8f 2024-09-14T22:54:12.246 INFO:teuthology.orchestra.run.smithi076.stdout:3268 punch from 0x9588e5 to 0x960ca3, (0x83be bytes) 2024-09-14T22:54:12.246 INFO:teuthology.orchestra.run.smithi076.stdout:3269 punch from 0x83e3cc to 0x84bce6, (0xd91a bytes) 2024-09-14T22:54:12.253 INFO:teuthology.orchestra.run.smithi076.stdout:3270 trunc from 0xa0af8f to 0x6894d43 2024-09-14T22:54:12.259 INFO:teuthology.orchestra.run.smithi076.stdout:3271 read 0x533ab55 thru 0x534943a (0xe8e6 bytes) 2024-09-14T22:54:12.259 INFO:teuthology.orchestra.run.smithi076.stdout:3272 read 0x2fec403 thru 0x2ff4073 (0x7c71 bytes) 2024-09-14T22:54:12.259 INFO:teuthology.orchestra.run.smithi076.stdout:3275 writesame 0xbd784f thru 0xbdc39e data_size 0x28(0x4b50 bytes) 2024-09-14T22:54:12.262 INFO:teuthology.orchestra.run.smithi076.stdout:3276 writesame 0x4a4108b thru 0x4a43490 data_size 0x6(0x2406 bytes) 2024-09-14T22:54:12.264 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:12.263+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:12.266 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.273 INFO:teuthology.orchestra.run.smithi076.stdout:3278 writesame 0x79bff15 thru 0x79c9578 data_size 0x37(0x9664 bytes) 2024-09-14T22:54:12.276 INFO:teuthology.orchestra.run.smithi076.stdout:3280 trunc from 0x79c9579 to 0x4fee6f5 2024-09-14T22:54:12.282 INFO:teuthology.orchestra.run.smithi076.stdout:3283 read 0xa5a857 thru 0xa5f651 (0x4dfb bytes) 2024-09-14T22:54:12.282 INFO:teuthology.orchestra.run.smithi076.stdout:3285 read 0x4782e9e thru 0x478922d (0x6390 bytes) 2024-09-14T22:54:12.282 INFO:teuthology.orchestra.run.smithi076.stdout:3286 write 0x4a95785 thru 0x4aa05e8 (0xae64 bytes) 2024-09-14T22:54:12.296 INFO:teuthology.orchestra.run.smithi076.stdout:3287 writesame 0x5159a87 thru 0x51660bf data_size 0x33(0xc639 bytes) 2024-09-14T22:54:12.296 INFO:teuthology.orchestra.run.smithi076.stdout:3288 read 0x4f2fcf thru 0x4f82e2 (0x5314 bytes) 2024-09-14T22:54:12.296 INFO:teuthology.orchestra.run.smithi076.stdout:3289 writesame 0x1be41d2 thru 0x1bf27ba data_size 0x17(0xe5e9 bytes) 2024-09-14T22:54:12.304 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.312 INFO:teuthology.orchestra.run.smithi076.stdout:3292 write 0x783b730 thru 0x783ddfc (0x26cd bytes) 2024-09-14T22:54:12.321 INFO:teuthology.orchestra.run.smithi076.stdout:3293 trunc from 0x783ddfd to 0xb84ed7d 2024-09-14T22:54:12.327 INFO:teuthology.orchestra.run.smithi076.stdout:3295 trunc from 0xb84ed7d to 0xc151205 2024-09-14T22:54:12.328 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:12.326+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:12.331 INFO:teuthology.orchestra.run.smithi076.stdout:3296 writesame 0x4147351 thru 0x4155730 data_size 0x10(0xe3e0 bytes) 2024-09-14T22:54:12.335 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:12.335 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:12.340 INFO:teuthology.orchestra.run.smithi076.stdout:3297 writesame 0x75d25ec thru 0x75d582b data_size 0x10(0x3240 bytes) 2024-09-14T22:54:12.344 INFO:teuthology.orchestra.run.smithi076.stdout:3298 punch from 0x5325f9d to 0x5334cf3, (0xed56 bytes) 2024-09-14T22:54:12.344 INFO:teuthology.orchestra.run.smithi076.stdout:3299 writesame 0x8cb4914 thru 0x8cc38a9 data_size 0x5b(0xef96 bytes) 2024-09-14T22:54:12.345 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:12.352 INFO:teuthology.orchestra.run.smithi076.stdout:3300 read 0x1829f58 thru 0x1835925 (0xb9ce bytes) 2024-09-14T22:54:12.361 INFO:teuthology.orchestra.run.smithi076.stdout:3301 write 0xe47ad2f thru 0xe47edc3 (0x4095 bytes) 2024-09-14T22:54:12.364 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:12.365 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:12.365 INFO:teuthology.orchestra.run.smithi076.stdout:3302 writesame 0xd446947 thru 0xd447d86 data_size 0x40(0x1440 bytes) 2024-09-14T22:54:12.369 INFO:teuthology.orchestra.run.smithi076.stdout:3305 read 0xdc863b3 thru 0xdc8d881 (0x74cf bytes) 2024-09-14T22:54:12.369 INFO:teuthology.orchestra.run.smithi076.stdout:3306 read 0x6bebd55 thru 0x6bf87c7 (0xca73 bytes) 2024-09-14T22:54:12.375 INFO:teuthology.orchestra.run.smithi076.stdout:3307 write 0xe5e47a6 thru 0xe5f3f9d (0xf7f8 bytes) 2024-09-14T22:54:12.381 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:12.381 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:12.383 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.383 INFO:teuthology.orchestra.run.smithi076.stdout:3309 write 0xc718a92 thru 0xc72518d (0xc6fc bytes) 2024-09-14T22:54:12.387 INFO:teuthology.orchestra.run.smithi076.stdout:3313 write 0xc20812b thru 0xc2162bc (0xe192 bytes) 2024-09-14T22:54:12.393 INFO:teuthology.orchestra.run.smithi076.stdout:3315 punch from 0x407acca to 0x4085ca7, (0xafdd bytes) 2024-09-14T22:54:12.393 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.394 INFO:teuthology.orchestra.run.smithi076.stdout:3317 write 0xc9d579d thru 0xc9de09a (0x88fe bytes) 2024-09-14T22:54:12.399 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:12.399 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:12.404 INFO:teuthology.orchestra.run.smithi076.stdout:3318 read 0xe2c8c72 thru 0xe2d7565 (0xe8f4 bytes) 2024-09-14T22:54:12.404 INFO:teuthology.orchestra.run.smithi076.stdout:3319 punch from 0x62469eb to 0x62544b4, (0xdac9 bytes) 2024-09-14T22:54:12.405 INFO:teuthology.orchestra.run.smithi076.stdout:3320 read 0xa919189 thru 0xa920ef0 (0x7d68 bytes) 2024-09-14T22:54:12.405 INFO:teuthology.orchestra.run.smithi076.stdout:3321 read 0xb5b9ebb thru 0xb5be12d (0x4273 bytes) 2024-09-14T22:54:12.405 INFO:teuthology.orchestra.run.smithi076.stdout:3325 writesame 0x8fe9608 thru 0x8ff48e2 data_size 0x1f(0xb2db bytes) 2024-09-14T22:54:12.409 INFO:teuthology.orchestra.run.smithi076.stdout:3326 punch from 0x21c16ae to 0x21d0c8c, (0xf5de bytes) 2024-09-14T22:54:12.409 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.409 INFO:teuthology.orchestra.run.smithi076.stdout:3331 read 0xbbdfdce thru 0xbbe503a (0x526d bytes) 2024-09-14T22:54:12.409 INFO:teuthology.orchestra.run.smithi076.stdout:3332 read 0x6ed4c8d thru 0x6ed576c (0xae0 bytes) 2024-09-14T22:54:12.415 INFO:teuthology.orchestra.run.smithi076.stdout:3333 writesame 0xeaf2b6f thru 0xeb0207f data_size 0x1(0xf511 bytes) 2024-09-14T22:54:12.415 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.415 INFO:teuthology.orchestra.run.smithi076.stdout:3338 write 0xd39ae75 thru 0xd3aa777 (0xf903 bytes) 2024-09-14T22:54:12.417 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:12.418 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:12.421 INFO:teuthology.orchestra.run.smithi076.stdout:3339 read 0x862c0f0 thru 0x8635b0d (0x9a1e bytes) 2024-09-14T22:54:12.429 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:12.426+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:12.430 INFO:teuthology.orchestra.run.smithi076.stdout:3340 trunc from 0xeb02080 to 0x6de4081 2024-09-14T22:54:12.437 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:12.437 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:12.445 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.445 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.445 INFO:teuthology.orchestra.run.smithi076.stdout:3343 punch from 0x78d9a3 to 0x7988ab, (0xaf08 bytes) 2024-09-14T22:54:12.445 INFO:teuthology.orchestra.run.smithi076.stdout:3344 write 0x6c62233 thru 0x6c67002 (0x4dd0 bytes) 2024-09-14T22:54:12.449 INFO:teuthology.orchestra.run.smithi076.stdout:3345 trunc from 0x6de4081 to 0x6049e17 2024-09-14T22:54:12.459 INFO:teuthology.orchestra.run.smithi076.stdout:3346 read 0x2fef646 thru 0x2ffb1dd (0xbb98 bytes) 2024-09-14T22:54:12.459 INFO:teuthology.orchestra.run.smithi076.stdout:3347 read 0x3691009 thru 0x369ef9a (0xdf92 bytes) 2024-09-14T22:54:12.459 INFO:teuthology.orchestra.run.smithi076.stdout:3349 read 0x5454870 thru 0x5459ed7 (0x5668 bytes) 2024-09-14T22:54:12.459 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.459 INFO:teuthology.orchestra.run.smithi076.stdout:3352 writesame 0x1194e1c thru 0x11a0196 data_size 0x1(0xb37b bytes) 2024-09-14T22:54:12.486 INFO:teuthology.orchestra.run.smithi076.stdout:3353 write 0x4f48683 thru 0x4f565af (0xdf2d bytes) 2024-09-14T22:54:12.491 INFO:teuthology.orchestra.run.smithi076.stdout:3354 punch from 0x37a7ae to 0x38a0c7, (0xf919 bytes) 2024-09-14T22:54:12.491 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.501 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.417848738000004, 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:54:12.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.417630842000001, 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.507 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:54:12.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.414947196, 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:54:12.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.414788780999999, 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:12.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:54:12.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.524 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.414761515000002, 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.525 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:54:12.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.414554373999998, 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.414520109999998, 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:54:12.534 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:54:12.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:54:12.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.412888064000001, 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:54:12.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.412863452000003, 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.543 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:12.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:12.547 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.412729165999998, 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:12.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.551 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.411679458000002, 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.552 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.411577272999999, 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.556 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.560 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.411553369000003, 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.561 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:54:12.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.411519849000001, 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:54:12.565 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:54:12.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:54:12.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.411365203000003, 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.574 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.411324028999999, 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:54:12.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:54:12.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.579 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.411295869, 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:12.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:54:12.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.411106478000001, 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.584 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:54:12.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.495016651, 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.494996819999997, 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:12.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:12.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:54:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:12.598 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:12.600 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3357 write 0xb8f05d2 thru 0xb8f21c8 (0x1bf7 bytes) 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3361 read 0x27cc792 thru 0x27d97ce (0xd03d bytes) 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3362 write 0x4acc993 thru 0x4ad5ed2 (0x9540 bytes) 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3363 write 0x5f0100c thru 0x5f09e68 (0x8e5d bytes) 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3364 trunc from 0xb8f21c9 to 0xe245ebf 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3365 read 0xc3b33c5 thru 0xc3bd04c (0x9c88 bytes) 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3366 punch from 0x343acb1 to 0x3441aaa, (0x6df9 bytes) 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3368 punch from 0x53aa2df to 0x53ae48b, (0x41ac bytes) 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout:3370 read 0x9c67af2 thru 0x9c6e3aa (0x68b9 bytes) 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3371 read 0x99f1b7e thru 0x99f2898 (0xd1b bytes) 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3372 read 0x2e15f9d thru 0x2e2527e (0xf2e2 bytes) 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3375 trunc from 0xe245ebf to 0x5e3b3c3 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3380 read 0x3a57e0d thru 0x3a63aed (0xbce1 bytes) 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3382 punch from 0x4ba2233 to 0x4baec97, (0xca64 bytes) 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3384 write 0x5b96858 thru 0x5b9cb0e (0x62b7 bytes) 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3385 read 0x292c03e thru 0x293afe0 (0xefa3 bytes) 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3386 writesame 0x62eba7a thru 0x62ee14e data_size 0x1(0x26d5 bytes) 2024-09-14T22:54:12.604 INFO:teuthology.orchestra.run.smithi076.stdout:3387 read 0x4d6cb9f thru 0x4d7286e (0x5cd0 bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3389 write 0x53b44b0 thru 0x53ba0f0 (0x5c41 bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3390 punch from 0x5f40bc6 to 0x5f48ffc, (0x8436 bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3391 read 0x2408596 thru 0x2408fa3 (0xa0e bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3393 writesame 0x845fe37 thru 0x8468aa7 data_size 0x1(0x8c71 bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3394 read 0x6f477ae thru 0x6f5637f (0xebd2 bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3395 write 0xa878c85 thru 0xa883d8d (0xb109 bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3397 read 0x47b1b9e thru 0x47bb6e5 (0x9b48 bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3398 write 0x28d12d7 thru 0x28d8ca7 (0x79d1 bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3399 write 0x710ca33 thru 0x710d1f0 (0x7be bytes) 2024-09-14T22:54:12.605 INFO:teuthology.orchestra.run.smithi076.stdout:3400 trunc from 0xa883d8e to 0xa7f930a 2024-09-14T22:54:12.606 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:12.597+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:12.606 INFO:teuthology.orchestra.run.smithi076.stdout:3401 trunc from 0xa7f930a to 0x560ed22 2024-09-14T22:54:12.617 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.617 INFO:teuthology.orchestra.run.smithi076.stdout:3404 writesame 0x476e21 thru 0x485507 data_size 0x1(0xe6e7 bytes) 2024-09-14T22:54:12.668 INFO:teuthology.orchestra.run.smithi076.stdout:3405 writesame 0x8410565 thru 0x84122e5 data_size 0xd(0x1d81 bytes) 2024-09-14T22:54:12.668 INFO:teuthology.orchestra.run.smithi076.stdout:3406 writesame 0x6302c46 thru 0x630f690 data_size 0x1(0xca4b bytes) 2024-09-14T22:54:12.697 INFO:teuthology.orchestra.run.smithi076.stdout:3407 writesame 0x554013a thru 0x5542ca4 data_size 0x41(0x2b6b bytes) 2024-09-14T22:54:12.700 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:12.698+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:12.703 INFO:teuthology.orchestra.run.smithi076.stdout:3408 trunc from 0x84122e6 to 0x69adab4 2024-09-14T22:54:12.712 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:12.712 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:12.712 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:12.712 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:12.713 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.713 INFO:teuthology.orchestra.run.smithi076.stdout:3410 read 0x581f562 thru 0x582005a (0xaf9 bytes) 2024-09-14T22:54:12.716 INFO:teuthology.orchestra.run.smithi076.stdout:3412 trunc from 0x69adab4 to 0x3b225f5 2024-09-14T22:54:12.722 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:12.730 INFO:teuthology.orchestra.run.smithi076.stdout:3414 read 0x2d49d79 thru 0x2d4feae (0x6136 bytes) 2024-09-14T22:54:12.747 INFO:teuthology.orchestra.run.smithi076.stdout:3416 writesame 0xd1aedd2 thru 0xd1b755a data_size 0x11(0x8789 bytes) 2024-09-14T22:54:12.760 INFO:teuthology.orchestra.run.smithi076.stdout:3418 trunc from 0xd1b755b to 0x20ba306 2024-09-14T22:54:12.791 INFO:teuthology.orchestra.run.smithi076.stdout:3419 write 0xe185363 thru 0xe18c93a (0x75d8 bytes) 2024-09-14T22:54:12.806 INFO:teuthology.orchestra.run.smithi076.stdout:3420 trunc from 0xe18c93b to 0x659f81b 2024-09-14T22:54:12.807 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:12.806+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:12.829 INFO:teuthology.orchestra.run.smithi076.stdout:3421 write 0xb023e6a thru 0xb0257f5 (0x198c bytes) 2024-09-14T22:54:12.832 INFO:teuthology.orchestra.run.smithi076.stdout:3423 writesame 0x2eb7200 thru 0x2ebfe77 data_size 0x7c(0x8c78 bytes) 2024-09-14T22:54:12.838 INFO:teuthology.orchestra.run.smithi076.stdout:3425 read 0x165749a thru 0x165e548 (0x70af bytes) 2024-09-14T22:54:12.838 INFO:teuthology.orchestra.run.smithi076.stdout:3427 read 0x24f5739 thru 0x250162b (0xbef3 bytes) 2024-09-14T22:54:12.839 INFO:teuthology.orchestra.run.smithi076.stdout:3428 read 0xa9ca4c3 thru 0xa9d72b0 (0xcdee bytes) 2024-09-14T22:54:12.839 INFO:teuthology.orchestra.run.smithi076.stdout:3429 writesame 0x33d720d thru 0x33e6900 data_size 0x91(0xf6f4 bytes) 2024-09-14T22:54:12.846 INFO:teuthology.orchestra.run.smithi076.stdout:3430 trunc from 0xb0257f6 to 0xa1218c4 2024-09-14T22:54:12.856 INFO:teuthology.orchestra.run.smithi076.stdout:3431 read 0x4185635 thru 0x418b31b (0x5ce7 bytes) 2024-09-14T22:54:12.856 INFO:teuthology.orchestra.run.smithi076.stdout:3432 punch from 0x50aa500 to 0x50aac0b, (0x70b bytes) 2024-09-14T22:54:12.856 INFO:teuthology.orchestra.run.smithi076.stdout:3433 writesame 0x18789e9 thru 0x187e5c2 data_size 0x3(0x5bda bytes) 2024-09-14T22:54:12.862 INFO:teuthology.orchestra.run.smithi076.stdout:3434 write 0x942719a thru 0x9435723 (0xe58a bytes) 2024-09-14T22:54:12.867 INFO:teuthology.orchestra.run.smithi076.stdout:3435 read 0x1234919 thru 0x123d9ad (0x9095 bytes) 2024-09-14T22:54:12.868 INFO:teuthology.orchestra.run.smithi076.stdout:3436 punch from 0xedbf40 to 0xee2a86, (0x6b46 bytes) 2024-09-14T22:54:12.868 INFO:teuthology.orchestra.run.smithi076.stdout:3437 writesame 0x9f25e21 thru 0x9f2731e data_size 0x1(0x14fe bytes) 2024-09-14T22:54:12.873 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:12.875 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:12.875 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:12.875 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:12.875 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:12.875 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:12.879 INFO:teuthology.orchestra.run.smithi076.stdout:3439 trunc from 0xa1218c4 to 0x4e1fdd7 2024-09-14T22:54:12.885 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:12.898 INFO:teuthology.orchestra.run.smithi076.stdout:3442 writesame 0x569f94d thru 0x56aeba1 data_size 0x3d(0xf255 bytes) 2024-09-14T22:54:12.908 INFO:teuthology.orchestra.run.smithi076.stdout:3443 writesame 0x824a579 thru 0x824bedd data_size 0xb(0x1965 bytes) 2024-09-14T22:54:12.908 INFO:teuthology.orchestra.run.smithi076.stdout:3444 punch from 0x81e3b07 to 0x81e768a, (0x3b83 bytes) 2024-09-14T22:54:12.908 INFO:teuthology.orchestra.run.smithi076.stdout:3446 trunc from 0x824bede to 0x864f2b8 2024-09-14T22:54:12.913 INFO:teuthology.orchestra.run.smithi076.stdout:truncating to largest ever: 0xedc642f 2024-09-14T22:54:12.924 INFO:teuthology.orchestra.run.smithi076.stdout:3447 trunc from 0x864f2b8 to 0xedc642f 2024-09-14T22:54:12.925 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:12.924+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:12.927 INFO:teuthology.orchestra.run.smithi076.stdout:3448 read 0x3cf9188 thru 0x3d051c7 (0xc040 bytes) 2024-09-14T22:54:12.936 INFO:teuthology.orchestra.run.smithi076.stdout:3451 trunc from 0xedc642f to 0x772eb44 2024-09-14T22:54:12.942 INFO:teuthology.orchestra.run.smithi076.stdout:3452 write 0x11db959 thru 0x11de0be (0x2766 bytes) 2024-09-14T22:54:12.957 INFO:teuthology.orchestra.run.smithi076.stdout:3454 write 0xc977941 thru 0xc9842cb (0xc98b bytes) 2024-09-14T22:54:12.961 INFO:teuthology.orchestra.run.smithi076.stdout:3455 write 0xa5cb104 thru 0xa5d0728 (0x5625 bytes) 2024-09-14T22:54:12.964 INFO:teuthology.orchestra.run.smithi076.stdout:3458 writesame 0x1ce3284 thru 0x1ce9d36 data_size 0xf(0x6ab3 bytes) 2024-09-14T22:54:12.977 INFO:teuthology.orchestra.run.smithi076.stdout:3459 write 0xe77deba thru 0xe78c464 (0xe5ab bytes) 2024-09-14T22:54:12.981 INFO:teuthology.orchestra.run.smithi076.stdout:3460 write 0x4f9460b thru 0x4fa1b61 (0xd557 bytes) 2024-09-14T22:54:12.986 INFO:teuthology.orchestra.run.smithi076.stdout:3462 punch from 0xa982d78 to 0xa991cfb, (0xef83 bytes) 2024-09-14T22:54:12.986 INFO:teuthology.orchestra.run.smithi076.stdout:3464 write 0xc0a3825 thru 0xc0a62bf (0x2a9b bytes) 2024-09-14T22:54:12.991 INFO:teuthology.orchestra.run.smithi076.stdout:3466 trunc from 0xe78c465 to 0xc4a584a 2024-09-14T22:54:13.001 INFO:teuthology.orchestra.run.smithi076.stdout:3469 write 0xa3cb4a6 thru 0xa3d0b8b (0x56e6 bytes) 2024-09-14T22:54:13.002 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:13.001+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:13.004 INFO:teuthology.orchestra.run.smithi076.stdout:3471 read 0x2d320ef thru 0x2d32375 (0x287 bytes) 2024-09-14T22:54:13.005 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.005 INFO:teuthology.orchestra.run.smithi076.stdout:3475 punch from 0x3788845 to 0x37926f7, (0x9eb2 bytes) 2024-09-14T22:54:13.005 INFO:teuthology.orchestra.run.smithi076.stdout:3477 punch from 0x3ce4b82 to 0x3cf347d, (0xe8fb bytes) 2024-09-14T22:54:13.005 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.005 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.012 INFO:teuthology.orchestra.run.smithi076.stdout:3482 trunc from 0xc4a584a to 0x60b6c5b 2024-09-14T22:54:13.024 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.025 INFO:teuthology.orchestra.run.smithi076.stdout:3484 read 0x532cf6 thru 0x5421cf (0xf4da bytes) 2024-09-14T22:54:13.025 INFO:teuthology.orchestra.run.smithi076.stdout:3485 punch from 0x1226715 to 0x122aae2, (0x43cd bytes) 2024-09-14T22:54:13.025 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.025 INFO:teuthology.orchestra.run.smithi076.stdout:3488 read 0x4bdba1b thru 0x4be40b0 (0x8696 bytes) 2024-09-14T22:54:13.035 INFO:teuthology.orchestra.run.smithi076.stdout:3490 trunc from 0x60b6c5b to 0xe62393f 2024-09-14T22:54:13.041 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:13.042 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:13.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:13.042 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:13.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.042 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:54:13.042 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:13.042 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 80.531831072000003, 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:13.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:54:13.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:54:13.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:54:13.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.667173216999998, 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.667067281000001, 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:54:13.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666992579999999, 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.058 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:54:13.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666907098999999, 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.063 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:13.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666760713000002, 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.067 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:13.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:54:13.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666710713000001, 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:54:13.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:54:13.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666547549000001, 2024-09-14T22:54:13.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:54:13.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666509239, 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.081 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:54:13.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:54:13.085 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.66637557, 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.086 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:13.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:54:13.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666295003000002, 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:13.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:54:13.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:13.095 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666198066, 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:54:13.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.666146511000001, 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.100 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:54:13.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.104 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.663021882000002, 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.105 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:54:13.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.109 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.662890269000002, 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:54:13.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:54:13.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 36.662603857000001, 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.114 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:54:13.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.118 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6045676029999996, 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.119 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:54:13.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:54:13.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6045277749999993, 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:54:13.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:54:13.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.6045117599999994, 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.131 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:54:13.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:54:13.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.5553834050000006, 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:54:13.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:54:13.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:54:13.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:13.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:13.144 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:13.146 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:13.147 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.147 INFO:teuthology.orchestra.run.smithi076.stdout:3492 read 0xca2f5df thru 0xca39985 (0xa3a7 bytes) 2024-09-14T22:54:13.147 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.147 INFO:teuthology.orchestra.run.smithi076.stdout:3495 trunc from 0xe62393f to 0xc5352a4 2024-09-14T22:54:13.147 INFO:teuthology.orchestra.run.smithi076.stdout:3496 write 0x749918a thru 0x74a5aa8 (0xc91f bytes) 2024-09-14T22:54:13.147 INFO:teuthology.orchestra.run.smithi076.stdout:3497 read 0x559fed thru 0x564c4b (0xac5f bytes) 2024-09-14T22:54:13.148 INFO:teuthology.orchestra.run.smithi076.stdout:3498 write 0x6c831fd thru 0x6c8c6e5 (0x94e9 bytes) 2024-09-14T22:54:13.148 INFO:teuthology.orchestra.run.smithi076.stdout:3499 trunc from 0xc5352a4 to 0x6606a1d 2024-09-14T22:54:13.148 INFO:teuthology.orchestra.run.smithi076.stdout:3500 read 0x556d04a thru 0x557c2c5 (0xf27c bytes) 2024-09-14T22:54:13.148 INFO:teuthology.orchestra.run.smithi076.stdout:3502 trunc from 0x6606a1d to 0x6abd0aa 2024-09-14T22:54:13.148 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.148 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:54:13.148 INFO:teuthology.orchestra.run.smithi076.stdout:3504 write 0x7a71e2f thru 0x7a7bd46 (0x9f18 bytes) 2024-09-14T22:54:13.150 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.150 INFO:teuthology.orchestra.run.smithi076.stdout:3506 write 0xab89591 thru 0xab92fdd (0x9a4d bytes) 2024-09-14T22:54:13.150 INFO:teuthology.orchestra.run.smithi076.stdout:3507 write 0xd9f41e3 thru 0xd9f8fcb (0x4de9 bytes) 2024-09-14T22:54:13.150 INFO:teuthology.orchestra.run.smithi076.stdout:3508 read 0x817e9d6 thru 0x818bf19 (0xd544 bytes) 2024-09-14T22:54:13.150 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.150 INFO:teuthology.orchestra.run.smithi076.stdout:3511 read 0x4c98847 thru 0x4ca0faa (0x8764 bytes) 2024-09-14T22:54:13.150 INFO:teuthology.orchestra.run.smithi076.stdout:3513 writesame 0xb2d6a5e thru 0xb2e3248 data_size 0x1(0xc7eb bytes) 2024-09-14T22:54:13.150 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:13.143+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:13.163 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.163 INFO:teuthology.orchestra.run.smithi076.stdout:3516 read 0x18d2bac thru 0x18e00ad (0xd502 bytes) 2024-09-14T22:54:13.164 INFO:teuthology.orchestra.run.smithi076.stdout:3517 read 0xb94c78d thru 0xb9550d8 (0x894c bytes) 2024-09-14T22:54:13.170 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:13.170 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:54:13.170 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T22:54:13.171 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:54:13.171 INFO:teuthology.orchestra.run.smithi076.stdout:3519 write 0xe3793b4 thru 0xe37f429 (0x6076 bytes) 2024-09-14T22:54:13.174 INFO:teuthology.orchestra.run.smithi076.stdout:3520 punch from 0x780b6ff to 0x780debc, (0x27bd bytes) 2024-09-14T22:54:13.175 INFO:teuthology.orchestra.run.smithi076.stdout:3521 write 0x1d519d6 thru 0x1d5bd6b (0xa396 bytes) 2024-09-14T22:54:13.177 INFO:teuthology.orchestra.run.smithi076.stdout:3522 punch from 0xdeb958a to 0xdebf2a2, (0x5d18 bytes) 2024-09-14T22:54:13.177 INFO:teuthology.orchestra.run.smithi076.stdout:3523 read 0x9770e89 thru 0x977eed6 (0xe04e bytes) 2024-09-14T22:54:13.177 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.177 INFO:teuthology.orchestra.run.smithi076.stdout:3526 write 0x8c9e23d thru 0x8caccd7 (0xea9b bytes) 2024-09-14T22:54:13.187 INFO:teuthology.orchestra.run.smithi076.stdout:3527 trunc from 0xe37f42a to 0xb70d2cf 2024-09-14T22:54:13.208 INFO:teuthology.orchestra.run.smithi076.stdout:3528 write 0xea57f5a thru 0xea639ec (0xba93 bytes) 2024-09-14T22:54:13.217 INFO:teuthology.orchestra.run.smithi076.stdout:3529 write 0x35b6816 thru 0x35b8d74 (0x255f bytes) 2024-09-14T22:54:13.223 INFO:teuthology.orchestra.run.smithi076.stdout:3532 write 0xb87e3b8 thru 0xb87e838 (0x481 bytes) 2024-09-14T22:54:13.225 INFO:teuthology.orchestra.run.smithi076.stdout:3534 write 0x2dfbf74 thru 0x2e0703f (0xb0cc bytes) 2024-09-14T22:54:13.230 INFO:teuthology.orchestra.run.smithi076.stdout:3535 write 0xb16655d thru 0xb17255a (0xbffe bytes) 2024-09-14T22:54:13.235 INFO:teuthology.orchestra.run.smithi076.stdout:3537 writesame 0x637661b thru 0x6378705 data_size 0x3(0x20eb bytes) 2024-09-14T22:54:13.241 INFO:teuthology.orchestra.run.smithi076.stdout:3538 write 0x159308e thru 0x1598914 (0x5887 bytes) 2024-09-14T22:54:13.242 INFO:teuthology.orchestra.run.smithi076.stdout:3540 write 0x897c89c thru 0x898bb7d (0xf2e2 bytes) 2024-09-14T22:54:13.246 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:13.245+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:13.247 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.247 INFO:teuthology.orchestra.run.smithi076.stdout:3545 write 0xb630f77 thru 0xb6318d5 (0x95f bytes) 2024-09-14T22:54:13.249 INFO:teuthology.orchestra.run.smithi076.stdout:3547 write 0x200c1a1 thru 0x200c6a2 (0x502 bytes) 2024-09-14T22:54:13.252 INFO:teuthology.orchestra.run.smithi076.stdout:3548 read 0x93eb8f6 thru 0x93f1eea (0x65f5 bytes) 2024-09-14T22:54:13.253 INFO:teuthology.orchestra.run.smithi076.stdout:3549 punch from 0x464aadc to 0x464bb05, (0x1029 bytes) 2024-09-14T22:54:13.253 INFO:teuthology.orchestra.run.smithi076.stdout:3550 writesame 0x89ddae8 thru 0x89e104d data_size 0x5(0x3566 bytes) 2024-09-14T22:54:13.257 INFO:teuthology.orchestra.run.smithi076.stdout:3551 punch from 0xe141637 to 0xe14ec58, (0xd621 bytes) 2024-09-14T22:54:13.257 INFO:teuthology.orchestra.run.smithi076.stdout:3552 read 0xe502b0d thru 0xe503da0 (0x1294 bytes) 2024-09-14T22:54:13.257 INFO:teuthology.orchestra.run.smithi076.stdout:3553 read 0xe96aaeb thru 0xe97102e (0x6544 bytes) 2024-09-14T22:54:13.258 INFO:teuthology.orchestra.run.smithi076.stdout:3554 read 0x86634bb thru 0x866aca1 (0x77e7 bytes) 2024-09-14T22:54:13.258 INFO:teuthology.orchestra.run.smithi076.stdout:3555 punch from 0xe7f964a to 0xe8041c7, (0xab7d bytes) 2024-09-14T22:54:13.258 INFO:teuthology.orchestra.run.smithi076.stdout:3557 punch from 0x7219b60 to 0x7227461, (0xd901 bytes) 2024-09-14T22:54:13.258 INFO:teuthology.orchestra.run.smithi076.stdout:3558 write 0x3f5b571 thru 0x3f62f93 (0x7a23 bytes) 2024-09-14T22:54:13.262 INFO:teuthology.orchestra.run.smithi076.stdout:3559 write 0x220411d thru 0x2212b64 (0xea48 bytes) 2024-09-14T22:54:13.266 INFO:teuthology.orchestra.run.smithi076.stdout:3561 writesame 0x440a5c0 thru 0x440adb9 data_size 0x1(0x7fa bytes) 2024-09-14T22:54:13.269 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:13.269 INFO:teuthology.orchestra.run.smithi076.stdout:3563 punch from 0xd382fab to 0xd38cde6, (0x9e3b bytes) 2024-09-14T22:54:13.269 INFO:teuthology.orchestra.run.smithi076.stdout:3566 writesame 0x5f30a30 thru 0x5f3c2e4 data_size 0x31(0xb8b5 bytes) 2024-09-14T22:54:13.273 INFO:teuthology.orchestra.run.smithi076.stdout:3568 clone 33 order 22 su 2097152 sc 12 2024-09-14T22:54:13.277 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:13.277 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:13.277 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:13.277 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:13.287 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:13.348 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:13.346+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:13.432 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:13.432 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:13.432 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:13.441 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:13.468 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:13.468 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:13.468 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 81, 2024-09-14T22:54:13.468 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:54:13.468 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:54:13.469 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:54:13.070412+0000", 2024-09-14T22:54:13.469 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:54:13.469 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:54:13.469 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:54:13.469 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5804032, 2024-09-14T22:54:13.469 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:54:13.469 INFO:teuthology.orchestra.run.smithi076.stdout: "nodeep-scrub", 2024-09-14T22:54:13.469 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:54:13.472 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:13.473 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:54:13.474 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:13.475 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:13.476 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:13.477 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.478 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:13.479 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:13.480 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:13.481 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:54:13.482 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:13.483 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:13.484 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 3, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 3, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.75, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2.25, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2.25, 2024-09-14T22:54:13.485 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:13.486 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:54:13.487 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "80", 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:13.488 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 34, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 80, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:13.489 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:13.490 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:13.491 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 2, 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 2, 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:13.492 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.5721282958984375, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:13.493 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:13.495 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:13.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:54:13.497 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.498 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.0684051513671875, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 58, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.499 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.500 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.501 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.502 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:54:13.503 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:13.504 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 62, 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:13.505 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:54:13.506 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:13.507 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:54:13.508 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 0.5237274169921875, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 62, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.510 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:13.512 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.513 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:54:13.515 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 24, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.516 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.517 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:13.519 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.520 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:13.521 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 50, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.522 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:13.523 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:13.524 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.525 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:54:13.527 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.528 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:13.529 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.530 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.532 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.533 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 75, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:13.534 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.536 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:13.537 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.538 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.539 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:13.540 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:13.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:13.542 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:54:13.543 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:13.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.544 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.544 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:54:13.544 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:13.544 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:13.544 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:13.544 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:13.544 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:13.545 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "2.6", 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: 5, 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:13.546 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.7", 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: 7, 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.547 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.8", 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: 3, 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [], 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [], 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:54:13.548 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [], 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:54:13.549 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.550 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:13.551 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:13.466+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:13.552 INFO:tasks.thrashosds.thrasher:j is {'epoch': 81, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:54:13.070412+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 34, 'snap_epoch': 80, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5721282958984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0684051513671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 0.5237274169921875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [5, 2]}, {'pgid': '3.0', 'osds': [6, 7]}, {'pgid': '3.7', 'osds': [7, 6]}, {'pgid': '3.8', 'osds': [3, 6]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:54:13.553 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:54:13.553 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.695192046999999, 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.554 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:54:13.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.557 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.558 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.694254457, 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.559 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:54:13.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.694163093999997, 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.563 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:54:13.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.693518208999997, 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.568 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:54:13.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:54:13.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:54:13.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.693030829999998, 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.572 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:54:13.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.576 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.692143555000001, 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.577 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.578 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.579 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.580 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.691551130999997, 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.581 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:54:13.583 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.584 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.585 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.690818315000001, 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.586 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:54:13.587 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.588 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:54:13.589 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:13.590 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.690719166000001, 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.591 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:54:13.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.593 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.594 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.690400707999999, 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:54:13.595 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.597 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.598 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:54:13.599 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.689891213999999, 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.600 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.603 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.604 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.689833354999998, 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:54:13.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.607 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.689040732000002, 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:54:13.609 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:54:13.611 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.612 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:54:13.613 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.688365292999997, 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.614 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.616 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:54:13.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.688231983000001, 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:54:13.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.621 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.622 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.687615059999999, 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.623 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:13.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.625 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.627 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.687548202999999, 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.628 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:13.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.687097391999998, 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.633 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.637 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.686977159000001, 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:54:13.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.685659031999997, 2024-09-14T22:54:13.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:13.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:13.644 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:54:13.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:13.647 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:13.649 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:13.649 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:13.646+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:13.750 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:13.747+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:13.796 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:13.797 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:13.797 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:13.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:13.890 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:13.888+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:13.951 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:13.951 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:13.951 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:13.951 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:13.951 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:13.952 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:13.950+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:13.960 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:13.979 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:13.980 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:54:13.992 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-14T22:54:13.078780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1401154848,"num_objects":1955,"num_object_clones":300,"num_object_copies":3910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1955,"num_whiteouts":1255,"num_read":192766,"num_read_kb":863053,"num_write":9240,"num_write_kb":502667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":24112926,"num_keys_recovered":18,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8740,"ondisk_log_size":8740,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":935260,"kb_used_data":639868,"kb_used_omap":12,"kb_used_meta":295091,"kb_avail":754039460,"statfs":{"total":773094113280,"available":772136407040,"internally_reserved":0,"allocated":655224832,"data_stored":762911856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":302173745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":130891546,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":10,"num_read":10594,"num_read_kb":59163,"num_write":638,"num_write_kb":19843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.283559"},"pg_stats":[{"pgid":"3.1f","version":"77'421","reported_seq":7355,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.058483+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:54:09.058483+0000","last_peered":"2024-09-14T22:54:09.058483+0000","last_clean":"2024-09-14T22:54:09.058483+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:54:09.058483+0000","last_undegraded":"2024-09-14T22:54:09.058483+0000","last_fullsized":"2024-09-14T22:54:09.058483+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:26:21.612551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":41167509,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":37,"num_read":8648,"num_read_kb":26616,"num_write":516,"num_write_kb":5583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"77'263","reported_seq":7229,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.053320+0000","last_change":"2024-09-14T22:53:32.831259+0000","last_active":"2024-09-14T22:54:09.053320+0000","last_peered":"2024-09-14T22:54:09.053320+0000","last_clean":"2024-09-14T22:54:09.053320+0000","last_became_active":"2024-09-14T22:53:32.751884+0000","last_became_peered":"2024-09-14T22:53:32.751884+0000","last_unstale":"2024-09-14T22:54:09.053320+0000","last_undegraded":"2024-09-14T22:54:09.053320+0000","last_fullsized":"2024-09-14T22:54:09.053320+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:25:50.327395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098261599999999996,"stat_sum":{"num_bytes":22714053,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":40,"num_read":7146,"num_read_kb":14067,"num_write":267,"num_write_kb":10253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":301575,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"77'212","reported_seq":7633,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.259955+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:54:09.259955+0000","last_peered":"2024-09-14T22:54:09.259955+0000","last_clean":"2024-09-14T22:54:09.259955+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:54:09.259955+0000","last_undegraded":"2024-09-14T22:54:09.259955+0000","last_fullsized":"2024-09-14T22:54:09.259955+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:39.771780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":54615098,"num_objects":70,"num_object_clones":13,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":42,"num_read":7780,"num_read_kb":34359,"num_write":188,"num_write_kb":32269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"78'307","reported_seq":1696,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.248015+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:54:09.248015+0000","last_peered":"2024-09-14T22:54:09.248015+0000","last_clean":"2024-09-14T22:54:09.248015+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:54:09.248015+0000","last_undegraded":"2024-09-14T22:54:09.248015+0000","last_fullsized":"2024-09-14T22:54:09.248015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:29:40.811922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":38097071,"num_objects":50,"num_object_clones":6,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":33,"num_read":2197,"num_read_kb":7709,"num_write":376,"num_write_kb":6082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"78'167","reported_seq":1159,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.505231+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:54:09.505231+0000","last_peered":"2024-09-14T22:54:09.505231+0000","last_clean":"2024-09-14T22:54:09.505231+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:54:09.505231+0000","last_undegraded":"2024-09-14T22:54:09.505231+0000","last_fullsized":"2024-09-14T22:54:09.505231+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:56.589753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":74919607,"num_objects":52,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":31,"num_read":990,"num_read_kb":26059,"num_write":179,"num_write_kb":17287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"78'281","reported_seq":11232,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.498051+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:54:09.498051+0000","last_peered":"2024-09-14T22:54:09.498051+0000","last_clean":"2024-09-14T22:54:09.498051+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:54:09.498051+0000","last_undegraded":"2024-09-14T22:54:09.498051+0000","last_fullsized":"2024-09-14T22:54:09.498051+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:34:00.514689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":55036611,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":43,"num_read":11781,"num_read_kb":48316,"num_write":233,"num_write_kb":28357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"78'243","reported_seq":14820,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.459025+0000","last_change":"2024-09-14T22:53:51.943155+0000","last_active":"2024-09-14T22:54:09.459025+0000","last_peered":"2024-09-14T22:54:09.459025+0000","last_clean":"2024-09-14T22:54:09.459025+0000","last_became_active":"2024-09-14T22:53:51.942663+0000","last_became_peered":"2024-09-14T22:53:51.942663+0000","last_unstale":"2024-09-14T22:54:09.459025+0000","last_undegraded":"2024-09-14T22:54:09.459025+0000","last_fullsized":"2024-09-14T22:54:09.459025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:20:36.456684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":74458666,"num_objects":74,"num_object_clones":12,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":48,"num_read":14956,"num_read_kb":89134,"num_write":219,"num_write_kb":37316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"78'212","reported_seq":1202,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.443419+0000","last_change":"2024-09-14T22:53:32.823044+0000","last_active":"2024-09-14T22:54:09.443419+0000","last_peered":"2024-09-14T22:54:09.443419+0000","last_clean":"2024-09-14T22:54:09.443419+0000","last_became_active":"2024-09-14T22:53:32.749192+0000","last_became_peered":"2024-09-14T22:53:32.749192+0000","last_unstale":"2024-09-14T22:54:09.443419+0000","last_undegraded":"2024-09-14T22:54:09.443419+0000","last_fullsized":"2024-09-14T22:54:09.443419+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:13:03.276417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062866099999999998,"stat_sum":{"num_bytes":30225723,"num_objects":58,"num_object_clones":5,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":42,"num_read":874,"num_read_kb":20409,"num_write":179,"num_write_kb":9265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1050624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"77'188","reported_seq":5419,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.386990+0000","last_change":"2024-09-14T22:53:38.882316+0000","last_active":"2024-09-14T22:54:09.386990+0000","last_peered":"2024-09-14T22:54:09.386990+0000","last_clean":"2024-09-14T22:54:09.386990+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:54:09.386990+0000","last_undegraded":"2024-09-14T22:54:09.386990+0000","last_fullsized":"2024-09-14T22:54:09.386990+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:50:56.882140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":29139254,"num_objects":51,"num_object_clones":5,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":30,"num_read":5544,"num_read_kb":27440,"num_write":167,"num_write_kb":25228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"73'232","reported_seq":8427,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.072641+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:54:09.072641+0000","last_peered":"2024-09-14T22:54:09.072641+0000","last_clean":"2024-09-14T22:54:09.072641+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:54:09.072641+0000","last_undegraded":"2024-09-14T22:54:09.072641+0000","last_fullsized":"2024-09-14T22:54:09.072641+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.663115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":57688558,"num_objects":59,"num_object_clones":10,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":38,"num_read":9282,"num_read_kb":39405,"num_write":200,"num_write_kb":13800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"77'163","reported_seq":8408,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.258391+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:54:09.258391+0000","last_peered":"2024-09-14T22:54:09.258391+0000","last_clean":"2024-09-14T22:54:09.258391+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:54:09.258391+0000","last_undegraded":"2024-09-14T22:54:09.258391+0000","last_fullsized":"2024-09-14T22:54:09.258391+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:14:13.960535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":31769268,"num_objects":49,"num_object_clones":7,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":35,"num_read":8886,"num_read_kb":41130,"num_write":118,"num_write_kb":12472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"78'335","reported_seq":7848,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.503743+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:54:09.503743+0000","last_peered":"2024-09-14T22:54:09.503743+0000","last_clean":"2024-09-14T22:54:09.503743+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:54:09.503743+0000","last_undegraded":"2024-09-14T22:54:09.503743+0000","last_fullsized":"2024-09-14T22:54:09.503743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:54:50.439861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":86547664,"num_objects":72,"num_object_clones":13,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":39,"num_read":8478,"num_read_kb":44508,"num_write":410,"num_write_kb":18195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.053316+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:54:09.053316+0000","last_peered":"2024-09-14T22:54:09.053316+0000","last_clean":"2024-09-14T22:54:09.053316+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:54:09.053316+0000","last_undegraded":"2024-09-14T22:54:09.053316+0000","last_fullsized":"2024-09-14T22:54:09.053316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"78'458","reported_seq":7024,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.439135+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:54:09.439135+0000","last_peered":"2024-09-14T22:54:09.439135+0000","last_clean":"2024-09-14T22:54:09.439135+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:54:09.439135+0000","last_undegraded":"2024-09-14T22:54:09.439135+0000","last_fullsized":"2024-09-14T22:54:09.439135+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:33:21.128353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":68260620,"num_objects":55,"num_object_clones":8,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":31,"num_read":8704,"num_read_kb":38989,"num_write":563,"num_write_kb":23286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.072533+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:54:09.072533+0000","last_peered":"2024-09-14T22:54:09.072533+0000","last_clean":"2024-09-14T22:54:09.072533+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:54:09.072533+0000","last_undegraded":"2024-09-14T22:54:09.072533+0000","last_fullsized":"2024-09-14T22:54:09.072533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"77'127","reported_seq":1093,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.064806+0000","last_change":"2024-09-14T22:54:07.044085+0000","last_active":"2024-09-14T22:54:09.064806+0000","last_peered":"2024-09-14T22:54:09.064806+0000","last_clean":"2024-09-14T22:54:09.064806+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:54:09.064806+0000","last_undegraded":"2024-09-14T22:54:09.064806+0000","last_fullsized":"2024-09-14T22:54:09.064806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:05:21.357763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":33508006,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":29,"num_read":788,"num_read_kb":8806,"num_write":129,"num_write_kb":13488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.064810+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:54:09.064810+0000","last_peered":"2024-09-14T22:54:09.064810+0000","last_clean":"2024-09-14T22:54:09.064810+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:54:09.064810+0000","last_undegraded":"2024-09-14T22:54:09.064810+0000","last_fullsized":"2024-09-14T22:54:09.064810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"77'435","reported_seq":12628,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.462347+0000","last_change":"2024-09-14T22:53:51.942879+0000","last_active":"2024-09-14T22:54:09.462347+0000","last_peered":"2024-09-14T22:54:09.462347+0000","last_clean":"2024-09-14T22:54:09.462347+0000","last_became_active":"2024-09-14T22:53:51.942267+0000","last_became_peered":"2024-09-14T22:53:51.942267+0000","last_unstale":"2024-09-14T22:54:09.462347+0000","last_undegraded":"2024-09-14T22:54:09.462347+0000","last_fullsized":"2024-09-14T22:54:09.462347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:02:27.617119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":30179459,"num_objects":66,"num_object_clones":9,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":41,"num_read":14008,"num_read_kb":35165,"num_write":581,"num_write_kb":17502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.057991+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:54:09.057991+0000","last_peered":"2024-09-14T22:54:09.057991+0000","last_clean":"2024-09-14T22:54:09.057991+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:54:09.057991+0000","last_undegraded":"2024-09-14T22:54:09.057991+0000","last_fullsized":"2024-09-14T22:54:09.057991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"77'190","reported_seq":13610,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.072718+0000","last_change":"2024-09-14T22:53:35.869981+0000","last_active":"2024-09-14T22:54:09.072718+0000","last_peered":"2024-09-14T22:54:09.072718+0000","last_clean":"2024-09-14T22:54:09.072718+0000","last_became_active":"2024-09-14T22:53:35.868554+0000","last_became_peered":"2024-09-14T22:53:35.868554+0000","last_unstale":"2024-09-14T22:54:09.072718+0000","last_undegraded":"2024-09-14T22:54:09.072718+0000","last_fullsized":"2024-09-14T22:54:09.072718+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:53:36.941988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":74165212,"num_objects":77,"num_object_clones":14,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":45,"num_read":11917,"num_read_kb":46462,"num_write":168,"num_write_kb":18378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":3794613,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"77'273","reported_seq":4790,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.060651+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:54:09.060651+0000","last_peered":"2024-09-14T22:54:09.060651+0000","last_clean":"2024-09-14T22:54:09.060651+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:54:09.060651+0000","last_undegraded":"2024-09-14T22:54:09.060651+0000","last_fullsized":"2024-09-14T22:54:09.060651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:12:49.991144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":50120084,"num_objects":72,"num_object_clones":14,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":49,"num_read":5837,"num_read_kb":52610,"num_write":239,"num_write_kb":29426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":133,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.072362+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:54:09.072362+0000","last_peered":"2024-09-14T22:54:09.072362+0000","last_clean":"2024-09-14T22:54:09.072362+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:54:09.072362+0000","last_undegraded":"2024-09-14T22:54:09.072362+0000","last_fullsized":"2024-09-14T22:54:09.072362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"77'165","reported_seq":900,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.387112+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:54:09.387112+0000","last_peered":"2024-09-14T22:54:09.387112+0000","last_clean":"2024-09-14T22:54:09.387112+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:54:09.387112+0000","last_undegraded":"2024-09-14T22:54:09.387112+0000","last_fullsized":"2024-09-14T22:54:09.387112+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:26.468896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":8995552,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":23,"num_read":952,"num_read_kb":7825,"num_write":184,"num_write_kb":10689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.062633+0000","last_change":"2024-09-14T22:53:32.745426+0000","last_active":"2024-09-14T22:54:09.062633+0000","last_peered":"2024-09-14T22:54:09.062633+0000","last_clean":"2024-09-14T22:54:09.062633+0000","last_became_active":"2024-09-14T22:53:32.745120+0000","last_became_peered":"2024-09-14T22:53:32.745120+0000","last_unstale":"2024-09-14T22:54:09.062633+0000","last_undegraded":"2024-09-14T22:54:09.062633+0000","last_fullsized":"2024-09-14T22:54:09.062633+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:06:49.621943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"78'411","reported_seq":6213,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.058445+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:54:09.058445+0000","last_peered":"2024-09-14T22:54:09.058445+0000","last_clean":"2024-09-14T22:54:09.058445+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:54:09.058445+0000","last_undegraded":"2024-09-14T22:54:09.058445+0000","last_fullsized":"2024-09-14T22:54:09.058445+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:05:05.721580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":20894497,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":36,"num_read":7490,"num_read_kb":19214,"num_write":557,"num_write_kb":9912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.064526+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:54:09.064526+0000","last_peered":"2024-09-14T22:54:09.064526+0000","last_clean":"2024-09-14T22:54:09.064526+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:54:09.064526+0000","last_undegraded":"2024-09-14T22:54:09.064526+0000","last_fullsized":"2024-09-14T22:54:09.064526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"78'202","reported_seq":5208,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.443095+0000","last_change":"2024-09-14T22:53:32.825251+0000","last_active":"2024-09-14T22:54:09.443095+0000","last_peered":"2024-09-14T22:54:09.443095+0000","last_clean":"2024-09-14T22:54:09.443095+0000","last_became_active":"2024-09-14T22:53:32.746422+0000","last_became_peered":"2024-09-14T22:53:32.746422+0000","last_unstale":"2024-09-14T22:54:09.443095+0000","last_undegraded":"2024-09-14T22:54:09.443095+0000","last_fullsized":"2024-09-14T22:54:09.443095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:24.673070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":52328614,"num_objects":68,"num_object_clones":10,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":47,"num_read":4867,"num_read_kb":28298,"num_write":161,"num_write_kb":27209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":57590,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":132,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.072370+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:54:09.072370+0000","last_peered":"2024-09-14T22:54:09.072370+0000","last_clean":"2024-09-14T22:54:09.072370+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:54:09.072370+0000","last_undegraded":"2024-09-14T22:54:09.072370+0000","last_fullsized":"2024-09-14T22:54:09.072370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":282,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.064608+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:54:09.064608+0000","last_peered":"2024-09-14T22:54:09.064608+0000","last_clean":"2024-09-14T22:54:09.064608+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:54:09.064608+0000","last_undegraded":"2024-09-14T22:54:09.064608+0000","last_fullsized":"2024-09-14T22:54:09.064608+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"78'372","reported_seq":5657,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.529883+0000","last_change":"2024-09-14T22:53:48.924907+0000","last_active":"2024-09-14T22:54:09.529883+0000","last_peered":"2024-09-14T22:54:09.529883+0000","last_clean":"2024-09-14T22:54:09.529883+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:54:09.529883+0000","last_undegraded":"2024-09-14T22:54:09.529883+0000","last_fullsized":"2024-09-14T22:54:09.529883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:27:40.059667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":46577372,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":46,"num_read":7457,"num_read_kb":20820,"num_write":338,"num_write_kb":12237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"78'213","reported_seq":855,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.523056+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:54:09.523056+0000","last_peered":"2024-09-14T22:54:09.523056+0000","last_clean":"2024-09-14T22:54:09.523056+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:54:09.523056+0000","last_undegraded":"2024-09-14T22:54:09.523056+0000","last_fullsized":"2024-09-14T22:54:09.523056+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:49:10.235614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":19481459,"num_objects":66,"num_object_clones":9,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":48,"num_read":990,"num_read_kb":2540,"num_write":191,"num_write_kb":2026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"78'283","reported_seq":1089,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.053382+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:54:09.053382+0000","last_peered":"2024-09-14T22:54:09.053382+0000","last_clean":"2024-09-14T22:54:09.053382+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:54:09.053382+0000","last_undegraded":"2024-09-14T22:54:09.053382+0000","last_fullsized":"2024-09-14T22:54:09.053382+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:45:53.861635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":25045056,"num_objects":50,"num_object_clones":5,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":34,"num_read":1757,"num_read_kb":6315,"num_write":317,"num_write_kb":3601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"77'246","reported_seq":1736,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.088525+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:54:09.088525+0000","last_peered":"2024-09-14T22:54:09.088525+0000","last_clean":"2024-09-14T22:54:09.088525+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:54:09.088525+0000","last_undegraded":"2024-09-14T22:54:09.088525+0000","last_fullsized":"2024-09-14T22:54:09.088525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:15:37.582136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":27231045,"num_objects":60,"num_object_clones":9,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":44,"num_read":2081,"num_read_kb":14833,"num_write":238,"num_write_kb":8317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"77'356","reported_seq":4431,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.238502+0000","last_change":"2024-09-14T22:53:35.869109+0000","last_active":"2024-09-14T22:54:09.238502+0000","last_peered":"2024-09-14T22:54:09.238502+0000","last_clean":"2024-09-14T22:54:09.238502+0000","last_became_active":"2024-09-14T22:53:35.868711+0000","last_became_peered":"2024-09-14T22:53:35.868711+0000","last_unstale":"2024-09-14T22:54:09.238502+0000","last_undegraded":"2024-09-14T22:54:09.238502+0000","last_fullsized":"2024-09-14T22:54:09.238502+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:49:01.123027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":30336419,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":44,"num_read":5109,"num_read_kb":19744,"num_write":375,"num_write_kb":16309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1667203,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"78'165","reported_seq":2958,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.392346+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:54:09.392346+0000","last_peered":"2024-09-14T22:54:09.392346+0000","last_clean":"2024-09-14T22:54:09.392346+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:54:09.392346+0000","last_undegraded":"2024-09-14T22:54:09.392346+0000","last_fullsized":"2024-09-14T22:54:09.392346+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:59.465558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":25019994,"num_objects":58,"num_object_clones":6,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":43,"num_read":2880,"num_read_kb":11345,"num_write":165,"num_write_kb":4663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"77'238","reported_seq":6118,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.252045+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:54:09.252045+0000","last_peered":"2024-09-14T22:54:09.252045+0000","last_clean":"2024-09-14T22:54:09.252045+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:54:09.252045+0000","last_undegraded":"2024-09-14T22:54:09.252045+0000","last_fullsized":"2024-09-14T22:54:09.252045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:23:21.586461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":52651150,"num_objects":60,"num_object_clones":11,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":39,"num_read":6845,"num_read_kb":38053,"num_write":196,"num_write_kb":19905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"78'167","reported_seq":3672,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.467305+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:54:09.467305+0000","last_peered":"2024-09-14T22:54:09.467305+0000","last_clean":"2024-09-14T22:54:09.467305+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:54:09.467305+0000","last_undegraded":"2024-09-14T22:54:09.467305+0000","last_fullsized":"2024-09-14T22:54:09.467305+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:59:27.461318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":25431811,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":38,"num_read":3513,"num_read_kb":20047,"num_write":145,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"78'278","reported_seq":8751,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.258006+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:54:09.258006+0000","last_peered":"2024-09-14T22:54:09.258006+0000","last_clean":"2024-09-14T22:54:09.258006+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:54:09.258006+0000","last_undegraded":"2024-09-14T22:54:09.258006+0000","last_fullsized":"2024-09-14T22:54:09.258006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:11:40.710272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":65822787,"num_objects":63,"num_object_clones":9,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":38,"num_read":10290,"num_read_kb":28479,"num_write":271,"num_write_kb":25166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"77'395","reported_seq":4780,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.232548+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:54:09.232548+0000","last_peered":"2024-09-14T22:54:09.232548+0000","last_clean":"2024-09-14T22:54:09.232548+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:54:09.232548+0000","last_undegraded":"2024-09-14T22:54:09.232548+0000","last_fullsized":"2024-09-14T22:54:09.232548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:29:41.472845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":52742162,"num_objects":58,"num_object_clones":10,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":5765,"num_read_kb":31472,"num_write":531,"num_write_kb":20023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"77'256","reported_seq":987,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.064702+0000","last_change":"2024-09-14T22:53:38.883182+0000","last_active":"2024-09-14T22:54:09.064702+0000","last_peered":"2024-09-14T22:54:09.064702+0000","last_clean":"2024-09-14T22:54:09.064702+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:54:09.064702+0000","last_undegraded":"2024-09-14T22:54:09.064702+0000","last_fullsized":"2024-09-14T22:54:09.064702+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:43:34.511685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":44011640,"num_objects":55,"num_object_clones":7,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":40,"num_read":1157,"num_read_kb":9345,"num_write":315,"num_write_kb":8594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"78'366","reported_seq":2875,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T22:54:09.458273+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:54:09.458273+0000","last_peered":"2024-09-14T22:54:09.458273+0000","last_clean":"2024-09-14T22:54:09.458273+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:54:09.458273+0000","last_undegraded":"2024-09-14T22:54:09.458273+0000","last_fullsized":"2024-09-14T22:54:09.458273+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:54:46.275911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":51382440,"num_objects":64,"num_object_clones":7,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":48,"num_read":3681,"num_read_kb":3439,"num_write":363,"num_write_kb":2111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1400564461,"num_objects":1951,"num_object_clones":300,"num_object_copies":3902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1951,"num_whiteouts":1255,"num_read":192650,"num_read_kb":862953,"num_write":9079,"num_write_kb":500249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":445,"num_bytes_recovered":24112926,"num_keys_recovered":18,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":647577600,"data_stored":759186674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8620,"ondisk_log_size":8620,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122544,"kb_used_data":83036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94249296,"statfs":{"total":96636764160,"available":96511279104,"internally_reserved":0,"allocated":85028864,"data_stored":98778978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96988,"kb_used_data":61644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94274852,"statfs":{"total":96636764160,"available":96537448448,"internally_reserved":0,"allocated":63123456,"data_stored":70540626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111072,"kb_used_data":75632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94260768,"statfs":{"total":96636764160,"available":96523026432,"internally_reserved":0,"allocated":77447168,"data_stored":85509212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":181772,"kb_used_data":138104,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94190068,"statfs":{"total":96636764160,"available":96450629632,"internally_reserved":0,"allocated":141418496,"data_stored":162600431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83324,"kb_used_data":47976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94288516,"statfs":{"total":96636764160,"available":96551440384,"internally_reserved":0,"allocated":49127424,"data_stored":53887421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542164,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71260,"kb_used_data":40064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94300580,"statfs":{"total":96636764160,"available":96563793920,"internally_reserved":0,"allocated":41025536,"data_stored":51172453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121088,"kb_used_data":85736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94250752,"statfs":{"total":96636764160,"available":96512770048,"internally_reserved":0,"allocated":87793664,"data_stored":114259934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147212,"kb_used_data":107676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94224628,"statfs":{"total":96636764160,"available":96486019072,"internally_reserved":0,"allocated":110260224,"data_stored":126162801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":109322240,"data_stored":125690705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86876160,"data_stored":113842907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40112128,"data_stored":50755446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48300032,"data_stored":53548770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":140705792,"data_stored":162361441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":76529664,"data_stored":85092185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":62210048,"data_stored":70123618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":83521536,"data_stored":97771602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:54:13.994 INFO:tasks.thrashosds.thrasher:Setting 3.2 to [4, 5] 2024-09-14T22:54:13.994 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2', '4', '5'] 2024-09-14T22:54:13.994 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2 4 5 2024-09-14T22:54:14.076 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:14.074+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:14.085 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:14.085 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:14.085 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:14.085 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:14.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.085 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.086628714, 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.086 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.087 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:54:14.088 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.089 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.086609676000002, 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.090 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:54:14.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.086573719999997, 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.095 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.096 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:54:14.097 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:54:14.098 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.086552599999997, 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.100 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:54:14.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.103 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.104 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.085345685999997, 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.107 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.108 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.083516238999998, 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.109 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:54:14.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:54:14.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:14.113 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.082961951999998, 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.114 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:54:14.117 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.080977234000002, 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.118 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:54:14.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:14.122 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.080624913999998, 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:14.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.126 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.080341470999997, 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:54:14.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:54:14.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:54:14.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.080053374000002, 2024-09-14T22:54:14.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:54:14.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.135 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.079678545999997, 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.136 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:14.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.140 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.079357731000002, 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:14.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.144 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.065672495999998, 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.145 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:54:14.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.149 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.065337687000003, 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.150 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:54:14.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.065007524999999, 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.154 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:54:14.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:54:14.158 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.064786738999999, 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.159 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:54:14.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.163 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.064517438000003, 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:54:14.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:54:14.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.064330341999998, 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:54:14.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:54:14.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:14.172 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.064012183000003, 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:14.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:54:14.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:54:14.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.177 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.177 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.177 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:14.177 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:14.177 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:14.178 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:14.179 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone32 from 0xea639ed (overlap 0x65a2a7) to 0x1453bb 2024-09-14T22:54:14.179 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:14.177+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:14.279 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-14T22:54:14.279 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:14.276+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:14.341 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:14.341 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:14.341 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:14.352 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:14.393 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:14.391+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:14.450 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:14.480 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:14.479+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:14.542 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:14.543 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:14.543 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:14.543 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:14.543 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:14.556 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:14.581 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:14.580+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:14.633 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:14.634 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:14.664 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:14.681 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:14.680+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:14.695 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:14.705 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:14.705 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:14.718 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:14.720 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:14.721 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:14.745 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:14.745 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:14.771 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:14.771 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:14.783 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:14.780+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:14.797 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:14.798 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:14.821 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:14.821 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:14.874 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:14.874 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:14.883 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:14.882+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:14.887 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:14.985 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:14.982+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:15.080 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:15.080 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:15.085 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:15.084+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:15.097 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:15.103 INFO:teuthology.orchestra.run.smithi076.stderr:set 3.2 pg_upmap mapping to [4,5] 2024-09-14T22:54:15.150 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:54:15.210 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:15.208+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:15.267 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:15.268 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:15.278 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:15.319 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:15.317+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:15.389 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:15.387+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:15.429 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:15.429 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:15.440 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:15.490 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:15.488+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:15.583 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:15.583 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:15.591 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:15.592 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:15.589+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:15.692 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:15.691+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:15.694 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:15.694 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:15.702 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:15.793 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:15.792+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:15.805 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:15.805 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:15.814 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:15.895 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:15.893+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:15.956 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:15.956 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:15.965 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:16.008 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:16.007+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:16.068 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:16.068 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:16.076 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:16.096 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:16.094+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:16.108 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:54:16.197 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:16.196+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:16.220 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:16.220 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:16.228 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:16.298 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:16.296+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:16.370 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:16.370 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:16.378 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:16.399 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:16.397+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:16.480 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:16.481 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:16.489 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:16.532 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:16.531+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:16.592 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:16.593 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:16.600 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:16.598+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:16.601 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:16.701 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:16.699+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:16.744 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:16.744 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:16.753 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:16.802 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:16.800+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:16.837 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:16.897 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:16.897 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:16.902 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:16.901+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:16.906 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:16.969 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:16.969 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:16.986 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:16.986 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:17.003 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:17.003 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:17.003 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:17.001+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:17.015 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:17.015 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:17.015 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:17.016 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:17.019 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:17.019 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:17.025 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:17.036 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:17.036 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:17.055 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:17.071 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:17.071 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:17.088 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:17.088 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:17.104 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:17.102+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:17.145 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:17.145 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:17.145 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:17.146 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:17.146 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:17.156 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:17.205 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:17.203+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:17.284 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.432891877000003, 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.285 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:17.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.431954287000003, 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:54:17.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:54:17.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:17.294 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.431862924000001, 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:17.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:54:17.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.431218039000001, 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.299 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:54:17.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:54:17.303 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.430730660000002, 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:17.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.429843384999998, 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:54:17.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.429250961000001, 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.313 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:54:17.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.428518144999998, 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.318 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:54:17.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.322 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.428418995999998, 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:54:17.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.428100538000002, 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.327 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.331 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.427591044000003, 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.332 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:54:17.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.336 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.427533185000001, 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.337 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.426740561999999, 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:54:17.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:54:17.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.426065123000001, 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.346 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:54:17.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:54:17.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.350 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.425931812999998, 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:54:17.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.425314890000003, 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.355 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:54:17.359 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.425248033000003, 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:54:17.360 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:17.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.364 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.424797222000002, 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.365 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:54:17.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.424676988999998, 2024-09-14T22:54:17.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:54:17.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.373 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.423358862000001, 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.374 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:54:17.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.378 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:17.380 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:17.380 INFO:teuthology.orchestra.run.smithi076.stdout:3569 trunc from 0xea639ed to 0x9fc7f4c 2024-09-14T22:54:17.381 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:17.377+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:17.399 INFO:teuthology.orchestra.run.smithi076.stdout:3570 write 0x7337e87 thru 0x7342d59 (0xaed3 bytes) 2024-09-14T22:54:17.403 INFO:teuthology.orchestra.run.smithi076.stdout:3573 read 0x899aac6 thru 0x899fd99 (0x52d4 bytes) 2024-09-14T22:54:17.403 INFO:teuthology.orchestra.run.smithi076.stdout:3574 writesame 0x1336eca thru 0x133bd65 data_size 0x56(0x4e9c bytes) 2024-09-14T22:54:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:3575 punch from 0x7a83766 to 0x7a84a7c, (0x1316 bytes) 2024-09-14T22:54:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:17.407 INFO:teuthology.orchestra.run.smithi076.stdout:3578 read 0x9168060 thru 0x916c6dc (0x467d bytes) 2024-09-14T22:54:17.413 INFO:teuthology.orchestra.run.smithi076.stdout:3579 write 0xb272c47 thru 0xb27e59d (0xb957 bytes) 2024-09-14T22:54:17.415 INFO:teuthology.orchestra.run.smithi076.stdout:3580 write 0x29118bc thru 0x29197a1 (0x7ee6 bytes) 2024-09-14T22:54:17.418 INFO:teuthology.orchestra.run.smithi076.stdout:3581 read 0x138e0f7 thru 0x1396da8 (0x8cb2 bytes) 2024-09-14T22:54:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:17.419 INFO:teuthology.orchestra.run.smithi076.stdout:3585 write 0x1e94fc3 thru 0x1e9b3f3 (0x6431 bytes) 2024-09-14T22:54:17.424 INFO:teuthology.orchestra.run.smithi076.stdout:3586 read 0x296018a thru 0x2963a57 (0x38ce bytes) 2024-09-14T22:54:17.424 INFO:teuthology.orchestra.run.smithi076.stdout:3587 writesame 0x9ff0bb0 thru 0x9ffbbab data_size 0xe(0xaffc bytes) 2024-09-14T22:54:17.427 INFO:teuthology.orchestra.run.smithi076.stdout:3590 clone 34 order 19 su 131072 sc 9 2024-09-14T22:54:17.480 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:17.478+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:17.505 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:17.505 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:17.505 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:17.505 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:17.513 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:17.581 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:17.579+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:17.656 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:17.656 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:17.656 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:17.657 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:17.657 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:17.665 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:17.708 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:17.706+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:17.773 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:17.773 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:17.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.774794202999999, 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.774 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.774775165000001, 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:54:17.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:54:17.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.774739209000003, 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.783 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:54:17.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:54:17.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.774718088999997, 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.788 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:54:17.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:54:17.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:54:17.793 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.793 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.793 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.773511175000003, 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:54:17.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:17.798 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.771681727999997, 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:17.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.771127440999997, 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.803 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:54:17.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:54:17.807 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.769142723000002, 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.808 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:54:17.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.812 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.768790402999997, 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.813 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:54:17.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.768506960000003, 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.817 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.819 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:54:17.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.768218863000001, 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.822 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.826 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.767844035000003, 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.827 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:54:17.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:17.831 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.767523220000001, 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:54:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.753837984999997, 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.836 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:54:17.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.840 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.753503176000002, 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.841 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:54:17.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:54:17.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.753173013999998, 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:17.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:54:17.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.752952227999998, 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.850 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:54:17.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.854 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.752682927000002, 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:54:17.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:54:17.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.752495830999997, 2024-09-14T22:54:17.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:17.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.863 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.752177672000002, 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:54:17.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:17.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.867 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.868 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:17.868 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:17.869 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:17.870 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone33 from 0xb27e59e (overlap 0x9fc7f4c) to 0x5fde5fa 2024-09-14T22:54:17.870 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:17.867+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:17.970 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:17.968+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:18.012 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:18.013 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:18.021 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:18.071 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:18.069+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:18.123 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:54:18.167 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:18.167 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:18.167 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:18.167 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:18.167 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:18.172 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:18.170+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:18.177 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:18.272 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:18.270+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:18.285 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:18.285 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8992 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'23, mlcod=33'21)", 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.376363953000002, 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0035453559999999999, 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906376+0000", 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.42e-06 2024-09-14T22:54:18.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906968+0000", 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00059252399999999998 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906990+0000", 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1498999999999999e-05 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906999+0000", 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2299999999999997e-06 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907058+0000", 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9505e-05 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907082+0000", 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3298999999999998e-05 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909874+0000", 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027926689999999998 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909919+0000", 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5210000000000003e-05 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8998 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'25, mlcod=33'21)", 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.374696018999998, 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003268272, 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908043+0000", 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908054+0000", 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1654999999999999e-05 2024-09-14T22:54:18.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908066+0000", 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1321000000000001e-05 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908077+0000", 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1781e-05 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908149+0000", 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1402000000000005e-05 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908170+0000", 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0746000000000001e-05 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911259+0000", 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030889839999999999 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911310+0000", 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1684999999999997e-05 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15153 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'58, mlcod=34'56)", 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.572521600999998, 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043627650000000002, 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.295 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710218+0000", 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.291e-06 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710957+0000", 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00073905599999999998 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710977+0000", 2024-09-14T22:54:18.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0506999999999999e-05 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710986+0000", 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0860000000000008e-06 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711048+0000", 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1130999999999995e-05 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711072+0000", 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4167000000000001e-05 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714533+0000", 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034611070000000002 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714579+0000", 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6419999999999999e-05 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.299 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15154 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'60, mlcod=34'56)", 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.571170846000001, 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00342618, 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711568+0000", 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0900000000000002e-07 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711574+0000", 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5119999999999997e-06 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711583+0000", 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7579999999999998e-06 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711589+0000", 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9619999999999998e-06 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711692+0000", 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000103237 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711719+0000", 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7124999999999999e-05 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714953+0000", 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032341240000000001 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714993+0000", 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9953000000000002e-05 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.074216606999997, 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.304 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:54:18.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:78175 3.4 e41/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 41'105, mlcod=40'102)", 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 50.316937652999997, 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0049595489999999997, 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965801+0000", 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.35e-07 2024-09-14T22:54:18.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965834+0000", 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3683999999999998e-05 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965846+0000", 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1620999999999999e-05 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965855+0000", 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1130000000000007e-06 2024-09-14T22:54:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965939+0000", 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4029000000000005e-05 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965963+0000", 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3529000000000001e-05 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970705+0000", 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.004742853 2024-09-14T22:54:18.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970760+0000", 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4484999999999998e-05 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:116229 3.1e e48/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 48'99, mlcod=48'97)", 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.978645751999998, 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034125309999999999, 2024-09-14T22:54:18.313 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304095+0000", 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3029999999999998e-06 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304126+0000", 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1374999999999997e-05 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304140+0000", 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4518000000000001e-05 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304150+0000", 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304213+0000", 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3366000000000003e-05 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304237+0000", 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4379000000000002e-05 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307471+0000", 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032338829999999999 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307505+0000", 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3552e-05 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.317 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.950554832999998, 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:54:18.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:54:18.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:133036 3.4 e54/24 3:22b239af:::rbd_data.1135a4365f9c.0000000000000001:head v 54'137, mlcod=53'135)", 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 36.012854883999999, 2024-09-14T22:54:18.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002821278, 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:54:18.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7100000000000001e-07 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270380+0000", 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00049641500000000005 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270393+0000", 2024-09-14T22:54:18.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3088000000000001e-05 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270402+0000", 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6199999999999994e-06 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270470+0000", 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7459999999999994e-05 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270498+0000", 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8214000000000001e-05 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272657+0000", 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021591499999999999 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272704+0000", 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7160000000000002e-05 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.326 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 29.018787362000001, 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:54:18.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:142014 3.1e e65/44 3:7ae60eba:::rbd_data.1135a7a7c51e.0000000000000000:head v 65'220, mlcod=65'217)", 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 24.092784076000001, 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0042481029999999996, 2024-09-14T22:54:18.331 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189955+0000", 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0979999999999999e-06 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191197+0000", 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012418710000000001 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191217+0000", 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.978e-05 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191226+0000", 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6500000000000008e-06 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191288+0000", 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1227000000000002e-05 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191312+0000", 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4088999999999999e-05 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.194153+0000", 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002841437 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.194202+0000", 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8950999999999999e-05 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:155203 3.4 3:23c245a1:::rbd_data.1135d0236bc5.000000000000001b:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:04.172813+0000", 2024-09-14T22:54:18.335 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.109924974, 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027271249999999999, 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 155203 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172813+0000", 2024-09-14T22:54:18.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172813+0000", 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172815+0000", 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4720000000000001e-06 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172822+0000", 2024-09-14T22:54:18.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4309999999999997e-06 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172836+0000", 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4105e-05 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172843+0000", 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9550000000000003e-06 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172900+0000", 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7469000000000002e-05 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172907+0000", 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6470000000000001e-06 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.174386+0000", 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001478739 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.174459+0000", 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3569e-05 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.175540+0000", 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010807379999999999 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.340 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:167397 3.9 e78/24 3:90d76597:::rbd_object_map.113519349ea7:head v 78'195, mlcod=78'192)", 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:09.407817+0000", 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.8749205960000008, 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003855163, 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407817+0000", 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407817+0000", 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407818+0000", 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407822+0000", 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9890000000000003e-06 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407830+0000", 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6159999999999996e-06 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407835+0000", 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407901+0000", 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5820000000000003e-05 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407914+0000", 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3393e-05 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.411634+0000", 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0037199400000000001 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.411673+0000", 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8645e-05 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172275 3.9 e82/24 3:91bc404c:::rbd_data.113598f99d1a.0000000000000008:head v 82'230, mlcod=81'225)", 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.171731+0000", 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.111006765, 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0028425109999999998, 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.345 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171731+0000", 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171731+0000", 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171732+0000", 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171742+0000", 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6390000000000004e-06 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171754+0000", 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228e-05 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171762+0000", 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.541e-06 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171815+0000", 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2176000000000003e-05 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171839+0000", 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4751000000000001e-05 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.174535+0000", 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0026956929999999999 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.174574+0000", 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8829999999999999e-05 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172286 3.1e e82/44 3:7ab3fec2:::rbd_data.113598f99d1a.0000000000000020:head v 82'288, mlcod=81'284)", 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.172802+0000", 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.1099359260000004, 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0037798530000000001, 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172802+0000", 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172802+0000", 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172803+0000", 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6400000000000002e-07 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172809+0000", 2024-09-14T22:54:18.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6379999999999998e-06 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172820+0000", 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0305999999999999e-05 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172824+0000", 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6720000000000003e-06 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172864+0000", 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9882000000000002e-05 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172878+0000", 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3637e-05 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.176541+0000", 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036630500000000002 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.176582+0000", 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1004000000000002e-05 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172283 3.9 e82/24 3:9284f145:::rbd_data.113598f99d1a.000000000000001c:head v 82'232, mlcod=81'225)", 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.172843+0000", 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.1098946759999997, 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003802728, 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.354 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172843+0000", 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172843+0000", 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.355 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172844+0000", 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172849+0000", 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2179999999999998e-06 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172859+0000", 2024-09-14T22:54:18.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0079e-05 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172866+0000", 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6220000000000003e-06 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172908+0000", 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1766e-05 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172924+0000", 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6251000000000001e-05 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.176577+0000", 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036533410000000001 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.176646+0000", 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8719000000000004e-05 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.358 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.1088411200000001, 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:54:18.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:54:18.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172306 3.4 3:201e3f56:::rbd_data.113598f99d1a.0000000000000067:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.179710+0000", 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.1030282380000003, 2024-09-14T22:54:18.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0037384850000000002, 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172306 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179710+0000", 2024-09-14T22:54:18.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179710+0000", 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179712+0000", 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2299999999999998e-06 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179718+0000", 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7049999999999996e-06 2024-09-14T22:54:18.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179733+0000", 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4905999999999999e-05 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179745+0000", 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2310000000000001e-05 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179816+0000", 2024-09-14T22:54:18.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1408999999999993e-05 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179898+0000", 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1323999999999996e-05 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.180052+0000", 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00015394599999999999 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.182195+0000", 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021436099999999998 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.183314+0000", 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001118747 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.183404+0000", 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9815000000000007e-05 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.183448+0000", 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4483000000000001e-05 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172324 3.4 3:213525dd:::rbd_data.113598f99d1a.00000000000000b5:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.0960511100000003, 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051499340000000001, 2024-09-14T22:54:18.369 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172324 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186688+0000", 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1829999999999999e-06 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186693+0000", 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4100000000000001e-06 2024-09-14T22:54:18.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186705+0000", 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2169e-05 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186712+0000", 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9789999999999996e-06 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186758+0000", 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6687000000000002e-05 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186839+0000", 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-05 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186980+0000", 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000141569 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191467+0000", 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0044866840000000003 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191734+0000", 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026690299999999998 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191793+0000", 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9447000000000002e-05 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191837+0000", 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3433000000000002e-05 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172329 3.4 3:23ce00f5:::rbd_data.113598f99d1a.00000000000000ea:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.189174+0000", 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.0935640710000003, 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051388140000000002, 2024-09-14T22:54:18.375 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172329 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189174+0000", 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189174+0000", 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189175+0000", 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2399999999999997e-07 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189179+0000", 2024-09-14T22:54:18.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.839e-06 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189189+0000", 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8900000000000002e-06 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189194+0000", 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.305e-06 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189244+0000", 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0052999999999997e-05 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189326+0000", 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2003999999999996e-05 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189494+0000", 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016826800000000001 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.194024+0000", 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0045300150000000001 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.194261+0000", 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023702799999999999 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.194290+0000", 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9159999999999999e-05 2024-09-14T22:54:18.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.194313+0000", 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2427999999999999e-05 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.381 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:18.383 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:18.383 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:18.380+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:18.483 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:18.481+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:18.526 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:18.526 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:18.526 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:18.526 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:18.534 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:18.584 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:18.583+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:18.677 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:18.678 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:18.678 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:18.678 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:18.678 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:18.685 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:18.684+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:18.686 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:18.740 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:54:18.786 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:18.784+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902988524999998, 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.810 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:54:18.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.813 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.814 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902967988999997, 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.815 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:54:18.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:18.819 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902937989999998, 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:54:18.823 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902906837000003, 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:54:18.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.828 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902884180999997, 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:54:18.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902863680000003, 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.833 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:54:18.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902843918999999, 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:54:18.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:54:18.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:54:18.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:18.842 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902823134000002, 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:18.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:54:18.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902790721999999, 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.847 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:18.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:54:18.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902769323000001, 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:54:18.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:18.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:18.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:54:18.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:54:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:54:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:54:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:18.859 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902707923999998, 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:54:18.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902668011000003, 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.864 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:54:18.868 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902638254999999, 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.869 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:54:18.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.873 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902608020000002, 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:54:18.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902583280999998, 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.878 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:54:18.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902559850999999, 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:54:18.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902464115000001, 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:54:18.887 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:54:18.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.891 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902435193000002, 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.892 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.902411868000002, 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:54:18.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 33.769714125, 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:18.901 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:54:18.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:18.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:54:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:54:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:18.906 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:18.906 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:18.907 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:18.908 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:18.905+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-14T22:54:18.916 INFO:teuthology.orchestra.run.smithi076.stdout:3591 punch from 0xa5d4ed4 to 0xa5e2852, (0xd97e bytes) 2024-09-14T22:54:18.919 INFO:teuthology.orchestra.run.smithi076.stdout:3592 punch from 0x4df5cf5 to 0x4e00c76, (0xaf81 bytes) 2024-09-14T22:54:18.925 INFO:teuthology.orchestra.run.smithi076.stdout:3593 write 0xb9b0add thru 0xb9b9329 (0x884d bytes) 2024-09-14T22:54:18.929 INFO:teuthology.orchestra.run.smithi076.stdout:3594 read 0x61483e4 thru 0x614e7a2 (0x63bf bytes) 2024-09-14T22:54:18.938 INFO:teuthology.orchestra.run.smithi076.stdout:3595 write 0xdf1b127 thru 0xdf25275 (0xa14f bytes) 2024-09-14T22:54:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:18.943 INFO:teuthology.orchestra.run.smithi076.stdout:3599 write 0x660741e thru 0x660c941 (0x5524 bytes) 2024-09-14T22:54:18.953 INFO:teuthology.orchestra.run.smithi076.stdout:3600 write 0xec07ece thru 0xec179bd (0xfaf0 bytes) 2024-09-14T22:54:18.956 INFO:teuthology.orchestra.run.smithi076.stdout:3602 write 0x986bcc6 thru 0x98714a0 (0x57db bytes) 2024-09-14T22:54:18.960 INFO:teuthology.orchestra.run.smithi076.stdout:3603 writesame 0xc65969 thru 0xc71eb7 data_size 0x3(0xc54f bytes) 2024-09-14T22:54:18.985 INFO:teuthology.orchestra.run.smithi076.stdout:3604 trunc from 0xec179be to 0x1d9a96c 2024-09-14T22:54:19.008 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:19.006+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:19.013 INFO:teuthology.orchestra.run.smithi076.stdout:3605 read 0x100957 thru 0x10c005 (0xb6af bytes) 2024-09-14T22:54:19.014 INFO:teuthology.orchestra.run.smithi076.stdout:3606 writesame 0x1b0fc1f thru 0x1b1d41a data_size 0x2e(0xd7fc bytes) 2024-09-14T22:54:19.026 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:19.026 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:19.027 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:19.035 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:19.038 INFO:teuthology.orchestra.run.smithi076.stdout:3609 write 0xea3743d thru 0xea46ac8 (0xf68c bytes) 2024-09-14T22:54:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:3610 read 0xc7fe736 thru 0xc807282 (0x8b4d bytes) 2024-09-14T22:54:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:3611 read 0xbe1d239 thru 0xbe24825 (0x75ed bytes) 2024-09-14T22:54:19.042 INFO:teuthology.orchestra.run.smithi076.stdout:3614 writesame 0x13eca23 thru 0x13f67a9 data_size 0x7(0x9d87 bytes) 2024-09-14T22:54:19.049 INFO:teuthology.orchestra.run.smithi076.stdout:3615 write 0x8fe2b6f thru 0x8feccf8 (0xa18a bytes) 2024-09-14T22:54:19.052 INFO:teuthology.orchestra.run.smithi076.stdout:3616 writesame 0x5b3d6a9 thru 0x5b3fed9 data_size 0x1(0x2831 bytes) 2024-09-14T22:54:19.059 INFO:teuthology.orchestra.run.smithi076.stdout:3617 read 0xd9d72e1 thru 0xd9e69a8 (0xf6c8 bytes) 2024-09-14T22:54:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:3618 read 0x8e4a30b thru 0x8e4ad07 (0x9fd bytes) 2024-09-14T22:54:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:3619 punch from 0x505ba40 to 0x5061226, (0x57e6 bytes) 2024-09-14T22:54:19.060 INFO:teuthology.orchestra.run.smithi076.stdout:3620 writesame 0x5825f01 thru 0x5826689 data_size 0x1(0x789 bytes) 2024-09-14T22:54:19.072 INFO:teuthology.orchestra.run.smithi076.stdout:3622 trunc from 0xea46ac9 to 0x7c75d55 2024-09-14T22:54:19.091 INFO:teuthology.orchestra.run.smithi076.stdout:3623 write 0x7eb710c thru 0x7ec2c49 (0xbb3e bytes) 2024-09-14T22:54:19.097 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:19.097 INFO:teuthology.orchestra.run.smithi076.stdout:3625 clone 35 order 18 su 32768 sc 10 2024-09-14T22:54:19.100 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:19.108 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:19.107+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:19.145 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:19.145 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:19.145 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:19.146 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:19.146 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:19.154 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:19.209 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:19.207+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:19.219 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:19.219 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:19.236 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:19.236 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:19.252 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:19.253 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:19.268 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:19.268 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:19.286 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:19.286 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:19.299 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:19.302 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:19.302 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:19.308 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:19.310 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:19.308+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:19.319 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:19.319 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:19.335 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:19.335 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:19.411 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:19.410+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:19.412 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:19.412 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:19.421 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:19.512 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:19.511+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:19.523 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:19.523 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:19.532 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:19.616 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:19.615+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:19.675 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:19.675 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:19.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:19.714 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:19.713+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:19.815 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:19.813+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:19.837 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:19.838 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:19.840 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone34 from 0x7ec2c4a (overlap 0x1d9a96c) to 0x2f5dbe 2024-09-14T22:54:19.849 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:19.933 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:19.932+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:20.005 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:20.005 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:20.011 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-14T22:54:20.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:20.017 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:20.015+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:20.117 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:20.116+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:20.126 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:20.126 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T22:54:20.139 INFO:teuthology.orchestra.run.smithi076.stdout:3626 trunc from 0x7ec2c4a to 0x3d5be9a 2024-09-14T22:54:20.163 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:20.163 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:20.163 INFO:teuthology.orchestra.run.smithi076.stdout:3627 trunc from 0x3d5be9a to 0x3a14f5a 2024-09-14T22:54:20.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:20.204 INFO:teuthology.orchestra.run.smithi076.stdout:3628 write 0xbfdaaa2 thru 0xbfddc3d (0x319c bytes) 2024-09-14T22:54:20.215 INFO:teuthology.orchestra.run.smithi076.stdout:3629 write 0xd803d0d thru 0xd80e8b8 (0xabac bytes) 2024-09-14T22:54:20.218 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:20.217+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:20.222 INFO:teuthology.orchestra.run.smithi076.stdout:3630 punch from 0x46c2ecb to 0x46cfa64, (0xcb99 bytes) 2024-09-14T22:54:20.222 INFO:teuthology.orchestra.run.smithi076.stdout:3631 writesame 0x692fdd6 thru 0x693ea36 data_size 0x17(0xec61 bytes) 2024-09-14T22:54:20.230 INFO:teuthology.orchestra.run.smithi076.stdout:3633 writesame 0x69bc95c thru 0x69caaff data_size 0xe(0xe1a4 bytes) 2024-09-14T22:54:20.249 INFO:teuthology.orchestra.run.smithi076.stdout:3634 trunc from 0xd80e8b9 to 0x4172831 2024-09-14T22:54:20.262 INFO:teuthology.orchestra.run.smithi076.stdout:3636 trunc from 0x4172831 to 0x5edf829 2024-09-14T22:54:20.280 INFO:teuthology.orchestra.run.smithi076.stdout:3637 write 0xb61661d thru 0xb62262c (0xc010 bytes) 2024-09-14T22:54:20.285 INFO:teuthology.orchestra.run.smithi076.stdout:3638 write 0x414027b thru 0x414bace (0xb854 bytes) 2024-09-14T22:54:20.288 INFO:teuthology.orchestra.run.smithi076.stdout:3639 writesame 0x25ed48e thru 0x25f510c data_size 0x9d(0x7c7f bytes) 2024-09-14T22:54:20.292 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:20.292 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:3640 punch from 0x248b3d4 to 0x2498142, (0xcd6e bytes) 2024-09-14T22:54:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:3643 read 0xa14b2ad thru 0xa14c435 (0x1189 bytes) 2024-09-14T22:54:20.293 INFO:teuthology.orchestra.run.smithi076.stdout:3644 write 0x407d306 thru 0x4080f84 (0x3c7f bytes) 2024-09-14T22:54:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:3648 read 0x4a99025 thru 0x4aa8781 (0xf75d bytes) 2024-09-14T22:54:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:3650 read 0xdcc30b thru 0xddc29c (0xff92 bytes) 2024-09-14T22:54:20.296 INFO:teuthology.orchestra.run.smithi076.stdout:3651 trunc from 0xb62262d to 0xb7878fc 2024-09-14T22:54:20.301 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:20.302 INFO:teuthology.orchestra.run.smithi076.stdout:3652 read 0x70c4902 thru 0x70c5673 (0xd72 bytes) 2024-09-14T22:54:20.302 INFO:teuthology.orchestra.run.smithi076.stdout:3653 write 0x9624c26 thru 0x9625d4b (0x1126 bytes) 2024-09-14T22:54:20.303 INFO:teuthology.orchestra.run.smithi076.stdout:3655 punch from 0x2214994 to 0x221f13d, (0xa7a9 bytes) 2024-09-14T22:54:20.304 INFO:teuthology.orchestra.run.smithi076.stdout:3656 writesame 0xde8634 thru 0xdeabd1 data_size 0x3(0x259e bytes) 2024-09-14T22:54:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:3657 punch from 0x268e001 to 0x268f933, (0x1932 bytes) 2024-09-14T22:54:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:3658 read 0x9aa2ee2 thru 0x9aa57ec (0x290b bytes) 2024-09-14T22:54:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:3659 punch from 0x4134b00 to 0x4143b21, (0xf021 bytes) 2024-09-14T22:54:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:3660 read 0x8ef907b thru 0x8efd474 (0x43fa bytes) 2024-09-14T22:54:20.309 INFO:teuthology.orchestra.run.smithi076.stdout:3661 write 0xb76dbe5 thru 0xb772e29 (0x5245 bytes) 2024-09-14T22:54:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:3663 read 0x9a0c496 thru 0x9a134ee (0x7059 bytes) 2024-09-14T22:54:20.312 INFO:teuthology.orchestra.run.smithi076.stdout:3664 write 0xa0b6d1f thru 0xa0bc7ea (0x5acc bytes) 2024-09-14T22:54:20.316 INFO:teuthology.orchestra.run.smithi076.stdout:3665 writesame 0xaf0335 thru 0xaf19f4 data_size 0x1(0x16c0 bytes) 2024-09-14T22:54:20.319 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:20.318+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:20.322 INFO:teuthology.orchestra.run.smithi076.stdout:3666 write 0x1d2db2b thru 0x1d33fa9 (0x647f bytes) 2024-09-14T22:54:20.328 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.328 INFO:teuthology.orchestra.run.smithi076.stdout:3668 writesame 0xefd5f1 thru 0xf0911f data_size 0x1(0xbb2f bytes) 2024-09-14T22:54:20.357 INFO:teuthology.orchestra.run.smithi076.stdout:3669 read 0x30d9710 thru 0x30e7d45 (0xe636 bytes) 2024-09-14T22:54:20.358 INFO:teuthology.orchestra.run.smithi076.stdout:3670 punch from 0x35ae3e4 to 0x35bb6fb, (0xd317 bytes) 2024-09-14T22:54:20.358 INFO:teuthology.orchestra.run.smithi076.stdout:3671 punch from 0x1c6153e to 0x1c6ac32, (0x96f4 bytes) 2024-09-14T22:54:20.359 INFO:teuthology.orchestra.run.smithi076.stdout:3672 trunc from 0xb7878fc to 0xa8f2c15 2024-09-14T22:54:20.373 INFO:teuthology.orchestra.run.smithi076.stdout:3673 writesame 0x2cd43b5 thru 0x2ce23a7 data_size 0x1(0xdff3 bytes) 2024-09-14T22:54:20.419 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:20.418+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:20.426 INFO:teuthology.orchestra.run.smithi076.stdout:3675 write 0xec817e7 thru 0xec8fbe6 (0xe400 bytes) 2024-09-14T22:54:20.429 INFO:teuthology.orchestra.run.smithi076.stdout:3676 write 0x1079ca0 thru 0x107e6b4 (0x4a15 bytes) 2024-09-14T22:54:20.434 INFO:teuthology.orchestra.run.smithi076.stdout:3677 read 0x48775b9 thru 0x487d805 (0x624d bytes) 2024-09-14T22:54:20.434 INFO:teuthology.orchestra.run.smithi076.stdout:3681 writesame 0xdd7b3d4 thru 0xdd845c3 data_size 0x28(0x91f0 bytes) 2024-09-14T22:54:20.438 INFO:teuthology.orchestra.run.smithi076.stdout:3683 writesame 0x29e880b thru 0x29eb6f7 data_size 0x1(0x2eed bytes) 2024-09-14T22:54:20.450 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:20.451 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:20.459 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:20.464 INFO:teuthology.orchestra.run.smithi076.stdout:3684 trunc from 0xec8fbe7 to 0x5f8625 2024-09-14T22:54:20.520 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:20.518+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:20.520 INFO:teuthology.orchestra.run.smithi076.stdout:3685 write 0x7bb5e74 thru 0x7bb95ba (0x3747 bytes) 2024-09-14T22:54:20.524 INFO:teuthology.orchestra.run.smithi076.stdout:3687 trunc from 0x7bb95bb to 0x7b0262d 2024-09-14T22:54:20.530 INFO:teuthology.orchestra.run.smithi076.stdout:3688 read 0x3689bb0 thru 0x36959ed (0xbe3e bytes) 2024-09-14T22:54:20.539 INFO:teuthology.orchestra.run.smithi076.stdout:3689 writesame 0x9b8f44c thru 0x9b9841c data_size 0x1(0x8fd1 bytes) 2024-09-14T22:54:20.539 INFO:teuthology.orchestra.run.smithi076.stdout:3691 punch from 0x5f3884e to 0x5f3cac2, (0x4274 bytes) 2024-09-14T22:54:20.540 INFO:teuthology.orchestra.run.smithi076.stdout:3692 writesame 0x68d4e6f thru 0x68d9b7f data_size 0x1(0x4d11 bytes) 2024-09-14T22:54:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:3693 read 0x17993b7 thru 0x179f542 (0x618c bytes) 2024-09-14T22:54:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.554 INFO:teuthology.orchestra.run.smithi076.stdout:3695 write 0x30d168c thru 0x30d659e (0x4f13 bytes) 2024-09-14T22:54:20.559 INFO:teuthology.orchestra.run.smithi076.stdout:3696 read 0x4776c2a thru 0x47818d6 (0xacad bytes) 2024-09-14T22:54:20.559 INFO:teuthology.orchestra.run.smithi076.stdout:3697 write 0x3c7d11c thru 0x3c81ce8 (0x4bcd bytes) 2024-09-14T22:54:20.575 INFO:teuthology.orchestra.run.smithi076.stdout:3699 writesame 0xd4c8a15 thru 0xd4d039e data_size 0x2(0x798a bytes) 2024-09-14T22:54:20.575 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:3701 read 0x9987137 thru 0x9989d2e (0x2bf8 bytes) 2024-09-14T22:54:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:3703 read 0x501b3e8 thru 0x501e015 (0x2c2e bytes) 2024-09-14T22:54:20.576 INFO:teuthology.orchestra.run.smithi076.stdout:3704 write 0xd22f1ce thru 0xd238e0d (0x9c40 bytes) 2024-09-14T22:54:20.587 INFO:teuthology.orchestra.run.smithi076.stdout:3705 trunc from 0xd4d039f to 0xaad3759 2024-09-14T22:54:20.597 INFO:teuthology.orchestra.run.smithi076.stdout:3707 punch from 0x26ec0ca to 0x26fae4c, (0xed82 bytes) 2024-09-14T22:54:20.598 INFO:teuthology.orchestra.run.smithi076.stdout:3709 read 0x96ed6e2 thru 0x96f1bfa (0x4519 bytes) 2024-09-14T22:54:20.598 INFO:teuthology.orchestra.run.smithi076.stdout:3710 writesame 0xa1129e thru 0xa19d0b data_size 0x3a(0x8a6e bytes) 2024-09-14T22:54:20.606 INFO:teuthology.orchestra.run.smithi076.stdout:3711 trunc from 0xaad3759 to 0xd0b4fa5 2024-09-14T22:54:20.610 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:20.610 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:20.612 INFO:teuthology.orchestra.run.smithi076.stdout:3713 writesame 0x43afe7a thru 0x43b7be3 data_size 0x3(0x7d6a bytes) 2024-09-14T22:54:20.619 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:20.620 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:20.619+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:20.622 INFO:teuthology.orchestra.run.smithi076.stdout:3714 read 0xa3c86ed thru 0xa3cb0a1 (0x29b5 bytes) 2024-09-14T22:54:20.622 INFO:teuthology.orchestra.run.smithi076.stdout:3715 write 0x792e9ae thru 0x793bbe2 (0xd235 bytes) 2024-09-14T22:54:20.637 INFO:teuthology.orchestra.run.smithi076.stdout:3717 trunc from 0xd0b4fa5 to 0x3b544fd 2024-09-14T22:54:20.661 INFO:teuthology.orchestra.run.smithi076.stdout:3718 write 0x991b5d2 thru 0x9928e8f (0xd8be bytes) 2024-09-14T22:54:20.668 INFO:teuthology.orchestra.run.smithi076.stdout:3720 trunc from 0x9928e90 to 0x74b439c 2024-09-14T22:54:20.683 INFO:teuthology.orchestra.run.smithi076.stdout:3721 trunc from 0x74b439c to 0x2b3246b 2024-09-14T22:54:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:3722 writesame 0xe1e6b2d thru 0xe1ee42d data_size 0x1(0x7901 bytes) 2024-09-14T22:54:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:3723 read 0x30f1610 thru 0x30f4ea1 (0x3892 bytes) 2024-09-14T22:54:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:3724 read 0x3e68cd3 thru 0x3e70cfd (0x802b bytes) 2024-09-14T22:54:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:3726 punch from 0x1f6bf3 to 0x1f9a1a, (0x2e27 bytes) 2024-09-14T22:54:20.712 INFO:teuthology.orchestra.run.smithi076.stdout:3729 write 0x54a2776 thru 0x54ab574 (0x8dff bytes) 2024-09-14T22:54:20.717 INFO:teuthology.orchestra.run.smithi076.stdout:3731 punch from 0x20f38b6 to 0x20ffa87, (0xc1d1 bytes) 2024-09-14T22:54:20.717 INFO:teuthology.orchestra.run.smithi076.stdout:3732 punch from 0x3b1c185 to 0x3b265f5, (0xa470 bytes) 2024-09-14T22:54:20.717 INFO:teuthology.orchestra.run.smithi076.stdout:3733 write 0x710af1b thru 0x7115b28 (0xac0e bytes) 2024-09-14T22:54:20.722 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:20.720+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:20.737 INFO:teuthology.orchestra.run.smithi076.stdout:3735 trunc from 0xe1ee42e to 0x4c7ef3 2024-09-14T22:54:20.741 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:20.741 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:20.752 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:20.756 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:54:20.773 INFO:teuthology.orchestra.run.smithi076.stdout:3737 write 0xb6129d thru 0xb689bd (0x7721 bytes) 2024-09-14T22:54:20.791 INFO:teuthology.orchestra.run.smithi076.stdout:3739 writesame 0x673301f thru 0x673e6fe data_size 0x58(0xb6e0 bytes) 2024-09-14T22:54:20.791 INFO:teuthology.orchestra.run.smithi076.stdout:3743 writesame 0x4201687 thru 0x420ddf1 data_size 0xbb(0xc76b bytes) 2024-09-14T22:54:20.796 INFO:teuthology.orchestra.run.smithi076.stdout:3747 read 0x5e103c0 thru 0x5e1d4b9 (0xd0fa bytes) 2024-09-14T22:54:20.797 INFO:teuthology.orchestra.run.smithi076.stdout:3748 writesame 0x2ea104 thru 0x2f736e data_size 0x1(0xd26b bytes) 2024-09-14T22:54:20.823 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:20.822+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:20.841 INFO:teuthology.orchestra.run.smithi076.stdout:3749 read 0x40528c1 thru 0x4053903 (0x1043 bytes) 2024-09-14T22:54:20.855 INFO:teuthology.orchestra.run.smithi076.stdout:3750 write 0xc2b871d thru 0xc2c216b (0x9a4f bytes) 2024-09-14T22:54:20.861 INFO:teuthology.orchestra.run.smithi076.stdout:3752 punch from 0x1ab9c94 to 0x1abd1d9, (0x3545 bytes) 2024-09-14T22:54:20.863 INFO:teuthology.orchestra.run.smithi076.stdout:3753 trunc from 0xc2c216c to 0xab4eb94 2024-09-14T22:54:20.875 INFO:teuthology.orchestra.run.smithi076.stdout:3754 write 0x862476d thru 0x863300f (0xe8a3 bytes) 2024-09-14T22:54:20.880 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:3758 read 0x2d75e3d thru 0x2d820cd (0xc291 bytes) 2024-09-14T22:54:20.881 INFO:teuthology.orchestra.run.smithi076.stdout:3762 writesame 0xa5f3dd6 thru 0xa5fd5fb data_size 0x5f(0x9826 bytes) 2024-09-14T22:54:20.884 INFO:teuthology.orchestra.run.smithi076.stdout:3763 write 0x6418aea thru 0x6421a73 (0x8f8a bytes) 2024-09-14T22:54:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:3764 punch from 0x969739d to 0x96990d8, (0x1d3b bytes) 2024-09-14T22:54:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:3766 read 0x5c46ec thru 0x5c4a9b (0x3b0 bytes) 2024-09-14T22:54:20.888 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:3768 read 0xa926a97 thru 0xa92f425 (0x898f bytes) 2024-09-14T22:54:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:3769 read 0xa356a32 thru 0xa36514b (0xe71a bytes) 2024-09-14T22:54:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:3771 write 0x522ff76 thru 0x523a6d2 (0xa75d bytes) 2024-09-14T22:54:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:20.889 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:20.893 INFO:teuthology.orchestra.run.smithi076.stdout:3772 read 0x1b632e9 thru 0x1b72e67 (0xfb7f bytes) 2024-09-14T22:54:20.893 INFO:teuthology.orchestra.run.smithi076.stdout:3773 write 0x117d9b3 thru 0x1180224 (0x2872 bytes) 2024-09-14T22:54:20.896 INFO:teuthology.orchestra.run.smithi076.stdout:3774 writesame 0x752c63c thru 0x7534149 data_size 0x13(0x7b0e bytes) 2024-09-14T22:54:20.901 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:20.901 INFO:teuthology.orchestra.run.smithi076.stdout:3775 writesame 0x780c652 thru 0x7812fee data_size 0x1(0x699d bytes) 2024-09-14T22:54:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:3776 trunc from 0xab4eb94 to 0x11a9f9a 2024-09-14T22:54:20.933 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:20.931+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:20.972 INFO:teuthology.orchestra.run.smithi076.stdout:3777 write 0xe5c1e53 thru 0xe5cdc7c (0xbe2a bytes) 2024-09-14T22:54:20.995 INFO:teuthology.orchestra.run.smithi076.stdout:3778 trunc from 0xe5cdc7d to 0x2a22b61 2024-09-14T22:54:21.005 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.005 INFO:teuthology.orchestra.run.smithi076.stdout:3781 read 0x250e23d thru 0x250f6c1 (0x1485 bytes) 2024-09-14T22:54:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:3782 read 0x26fb3a1 thru 0x26ff3a5 (0x4005 bytes) 2024-09-14T22:54:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:3783 read 0x2260bdc thru 0x2266f26 (0x634b bytes) 2024-09-14T22:54:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:3784 read 0x1789135 thru 0x178c637 (0x3503 bytes) 2024-09-14T22:54:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:3787 read 0x21c544 thru 0x2283e8 (0xbea5 bytes) 2024-09-14T22:54:21.006 INFO:teuthology.orchestra.run.smithi076.stdout:3788 punch from 0x1628442 to 0x1635da3, (0xd961 bytes) 2024-09-14T22:54:21.007 INFO:teuthology.orchestra.run.smithi076.stdout:3789 read 0x1f2985c thru 0x1f355e0 (0xbd85 bytes) 2024-09-14T22:54:21.007 INFO:teuthology.orchestra.run.smithi076.stdout:3790 read 0x286e734 thru 0x2877351 (0x8c1e bytes) 2024-09-14T22:54:21.016 INFO:teuthology.orchestra.run.smithi076.stdout:3792 writesame 0x5c6fc98 thru 0x5c7b878 data_size 0x1(0xbbe1 bytes) 2024-09-14T22:54:21.017 INFO:teuthology.orchestra.run.smithi076.stdout:3793 read 0x2cb75c9 thru 0x2cbeed9 (0x7911 bytes) 2024-09-14T22:54:21.023 INFO:teuthology.orchestra.run.smithi076.stdout:3794 write 0x68704c0 thru 0x687ac8e (0xa7cf bytes) 2024-09-14T22:54:21.025 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:21.023+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:21.028 INFO:teuthology.orchestra.run.smithi076.stdout:3796 read 0x25ff041 thru 0x26082ed (0x92ad bytes) 2024-09-14T22:54:21.028 INFO:teuthology.orchestra.run.smithi076.stdout:3797 read 0x1c3d7a4 thru 0x1c46d34 (0x9591 bytes) 2024-09-14T22:54:21.035 INFO:teuthology.orchestra.run.smithi076.stdout:3798 write 0x7968f08 thru 0x7976f9a (0xe093 bytes) 2024-09-14T22:54:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:3801 punch from 0x1b33b6d to 0x1b3b909, (0x7d9c bytes) 2024-09-14T22:54:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:3802 read 0x6a5719a thru 0x6a5e84f (0x76b6 bytes) 2024-09-14T22:54:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.047 INFO:teuthology.orchestra.run.smithi076.stdout:3804 trunc from 0x7976f9b to 0x183c6c0 2024-09-14T22:54:21.050 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:21.050 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:21.056 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.060 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:21.064 INFO:teuthology.orchestra.run.smithi076.stdout:3809 write 0x34050ab thru 0x340c1c8 (0x711e bytes) 2024-09-14T22:54:21.067 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.067 INFO:teuthology.orchestra.run.smithi076.stdout:3812 read 0xc2cca0 thru 0xc39adb (0xce3c bytes) 2024-09-14T22:54:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:3815 writesame 0x66a3624 thru 0x66aaaaa data_size 0x1(0x7487 bytes) 2024-09-14T22:54:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:3816 read 0x21840ca thru 0x218534a (0x1281 bytes) 2024-09-14T22:54:21.077 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.087 INFO:teuthology.orchestra.run.smithi076.stdout:3818 write 0xa7153c1 thru 0xa7227c0 (0xd400 bytes) 2024-09-14T22:54:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:3819 read 0x808712b thru 0x809684c (0xf722 bytes) 2024-09-14T22:54:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:3820 punch from 0x51f06d7 to 0x51f0d50, (0x679 bytes) 2024-09-14T22:54:21.092 INFO:teuthology.orchestra.run.smithi076.stdout:3821 punch from 0x95745f1 to 0x9576af1, (0x2500 bytes) 2024-09-14T22:54:21.101 INFO:teuthology.orchestra.run.smithi076.stdout:3822 trunc from 0xa7227c1 to 0x1f9e68e 2024-09-14T22:54:21.112 INFO:teuthology.orchestra.run.smithi076.stdout:3823 read 0x163832f thru 0x1638562 (0x234 bytes) 2024-09-14T22:54:21.121 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T22:54:21.125 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:21.124+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:21.127 INFO:teuthology.orchestra.run.smithi076.stdout:3824 write 0x8311cdb thru 0x831aea2 (0x91c8 bytes) 2024-09-14T22:54:21.132 INFO:teuthology.orchestra.run.smithi076.stdout:3825 punch from 0x78bbdd7 to 0x78c73c0, (0xb5e9 bytes) 2024-09-14T22:54:21.132 INFO:teuthology.orchestra.run.smithi076.stdout:3826 write 0x20b05e9 thru 0x20b3ddd (0x37f5 bytes) 2024-09-14T22:54:21.141 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T22:54:21.145 INFO:teuthology.orchestra.run.smithi076.stdout:3827 write 0xbe50c1c thru 0xbe50ca3 (0x88 bytes) 2024-09-14T22:54:21.153 INFO:teuthology.orchestra.run.smithi076.stdout:3828 write 0xc483fa0 thru 0xc491b45 (0xdba6 bytes) 2024-09-14T22:54:21.159 INFO:teuthology.orchestra.run.smithi076.stdout:3829 writesame 0x671516 thru 0x6730d1 data_size 0x32(0x1bbc bytes) 2024-09-14T22:54:21.162 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.162 INFO:teuthology.orchestra.run.smithi076.stdout:3831 write 0x5485cae thru 0x5492202 (0xc555 bytes) 2024-09-14T22:54:21.166 INFO:teuthology.orchestra.run.smithi076.stdout:3833 read 0x4b5fcbb thru 0x4b65cfb (0x6041 bytes) 2024-09-14T22:54:21.166 INFO:teuthology.orchestra.run.smithi076.stdout:3834 writesame 0x825536c thru 0x8255ee9 data_size 0x1(0xb7e bytes) 2024-09-14T22:54:21.171 INFO:teuthology.orchestra.run.smithi076.stdout:3836 writesame 0x794bc23 thru 0x7954ccb data_size 0x1(0x90a9 bytes) 2024-09-14T22:54:21.192 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:21.192 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:21.193 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.204 INFO:teuthology.orchestra.run.smithi076.stdout:3839 trunc from 0xc491b46 to 0x2fac6e8 2024-09-14T22:54:21.205 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:21.226 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:21.224+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:21.233 INFO:teuthology.orchestra.run.smithi076.stdout:3840 writesame 0xb9740e5 thru 0xb977ced data_size 0x2f(0x3c09 bytes) 2024-09-14T22:54:21.233 INFO:teuthology.orchestra.run.smithi076.stdout:3841 write 0x4db1e8d thru 0x4db9a20 (0x7b94 bytes) 2024-09-14T22:54:21.236 INFO:teuthology.orchestra.run.smithi076.stdout:3842 write 0x7313a42 thru 0x731b7a7 (0x7d66 bytes) 2024-09-14T22:54:21.240 INFO:teuthology.orchestra.run.smithi076.stdout:3843 read 0x956d620 thru 0x9577725 (0xa106 bytes) 2024-09-14T22:54:21.240 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.240 INFO:teuthology.orchestra.run.smithi076.stdout:3845 writesame 0x85e1b5d thru 0x85e5da3 data_size 0x13(0x4247 bytes) 2024-09-14T22:54:21.243 INFO:teuthology.orchestra.run.smithi076.stdout:3847 punch from 0x68d7da3 to 0x68e232a, (0xa587 bytes) 2024-09-14T22:54:21.245 INFO:teuthology.orchestra.run.smithi076.stdout:3849 trunc from 0xb977cee to 0xa3519e2 2024-09-14T22:54:21.254 INFO:teuthology.orchestra.run.smithi076.stdout:3852 trunc from 0xa3519e2 to 0x82df391 2024-09-14T22:54:21.263 INFO:teuthology.orchestra.run.smithi076.stdout:3854 write 0x794efc4 thru 0x79598a1 (0xa8de bytes) 2024-09-14T22:54:21.267 INFO:teuthology.orchestra.run.smithi076.stdout:3856 read 0x35d4668 thru 0x35e28c1 (0xe25a bytes) 2024-09-14T22:54:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:3858 read 0xfb0a7a thru 0xfb8362 (0x78e9 bytes) 2024-09-14T22:54:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:3860 punch from 0x6808830 to 0x68115d3, (0x8da3 bytes) 2024-09-14T22:54:21.268 INFO:teuthology.orchestra.run.smithi076.stdout:3861 write 0x1f8f7e4 thru 0x1f953bc (0x5bd9 bytes) 2024-09-14T22:54:21.279 INFO:teuthology.orchestra.run.smithi076.stdout:3862 trunc from 0x82df391 to 0xe7c1911 2024-09-14T22:54:21.285 INFO:teuthology.orchestra.run.smithi076.stdout:3863 writesame 0xc72fcc thru 0xc7ed11 data_size 0x7(0xbd46 bytes) 2024-09-14T22:54:21.291 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.291 INFO:teuthology.orchestra.run.smithi076.stdout:3865 write 0x3f06dc8 thru 0x3f1082b (0x9a64 bytes) 2024-09-14T22:54:21.302 INFO:teuthology.orchestra.run.smithi076.stdout:3866 write 0xee49905 thru 0xee49998 (0x94 bytes) 2024-09-14T22:54:21.305 INFO:teuthology.orchestra.run.smithi076.stdout:3867 read 0xd391f5a thru 0xd39abad (0x8c54 bytes) 2024-09-14T22:54:21.305 INFO:teuthology.orchestra.run.smithi076.stdout:3870 writesame 0x12da367 thru 0x12daf4a data_size 0x2(0xbe4 bytes) 2024-09-14T22:54:21.310 INFO:teuthology.orchestra.run.smithi076.stdout:3871 read 0xb593d1a thru 0xb596516 (0x27fd bytes) 2024-09-14T22:54:21.310 INFO:teuthology.orchestra.run.smithi076.stdout:3872 writesame 0x5c0e9a4 thru 0x5c1130b data_size 0x19(0x2968 bytes) 2024-09-14T22:54:21.321 INFO:teuthology.orchestra.run.smithi076.stdout:3874 writesame 0x41a3c53 thru 0x41aeae6 data_size 0x2(0xae94 bytes) 2024-09-14T22:54:21.327 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:21.326+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:21.337 INFO:teuthology.orchestra.run.smithi076.stdout:3875 writesame 0x6dcb5fa thru 0x6dcf420 data_size 0x1(0x3e27 bytes) 2024-09-14T22:54:21.346 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:21.347 INFO:teuthology.orchestra.run.smithi076.stdout:3877 write 0x48af858 thru 0x48b6faa (0x7753 bytes) 2024-09-14T22:54:21.353 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.354 INFO:teuthology.orchestra.run.smithi076.stdout:3880 punch from 0x9c4e694 to 0x9c53aca, (0x5436 bytes) 2024-09-14T22:54:21.354 INFO:teuthology.orchestra.run.smithi076.stdout:3881 write 0x93b1ba7 thru 0x93ba48f (0x88e9 bytes) 2024-09-14T22:54:21.360 INFO:teuthology.orchestra.run.smithi076.stdout:3883 read 0x8c59045 thru 0x8c6245a (0x9416 bytes) 2024-09-14T22:54:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.361 INFO:teuthology.orchestra.run.smithi076.stdout:3886 writesame 0x6d9b431 thru 0x6da61df data_size 0x1(0xadaf bytes) 2024-09-14T22:54:21.365 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:21.365 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:21.380 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:3887 punch from 0x102987e to 0x102f77e, (0x5f00 bytes) 2024-09-14T22:54:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:3888 punch from 0x47742a to 0x482a1f, (0xb5f5 bytes) 2024-09-14T22:54:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:3889 read 0xedceb9c thru 0xedd757b (0x89e0 bytes) 2024-09-14T22:54:21.390 INFO:teuthology.orchestra.run.smithi076.stdout:3891 write 0xd505420 thru 0xd50fd2f (0xa910 bytes) 2024-09-14T22:54:21.394 INFO:teuthology.orchestra.run.smithi076.stdout:3892 read 0xb77c168 thru 0xb781289 (0x5122 bytes) 2024-09-14T22:54:21.394 INFO:teuthology.orchestra.run.smithi076.stdout:3895 writesame 0x45b6ac1 thru 0x45b9d5c data_size 0x29(0x329c bytes) 2024-09-14T22:54:21.397 INFO:teuthology.orchestra.run.smithi076.stdout:3896 writesame 0x8a7f1e9 thru 0x8a813c3 data_size 0x1b(0x21db bytes) 2024-09-14T22:54:21.419 INFO:teuthology.orchestra.run.smithi076.stdout:3897 trunc from 0xee49999 to 0x129b777 2024-09-14T22:54:21.428 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:21.426+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:21.462 INFO:teuthology.orchestra.run.smithi076.stdout:3898 writesame 0xa659d7d thru 0xa66053d data_size 0x1(0x67c1 bytes) 2024-09-14T22:54:21.462 INFO:teuthology.orchestra.run.smithi076.stdout:3900 write 0x82fe63a thru 0x83054bb (0x6e82 bytes) 2024-09-14T22:54:21.468 INFO:teuthology.orchestra.run.smithi076.stdout:3901 writesame 0x652645f thru 0x652fb80 data_size 0xa(0x9722 bytes) 2024-09-14T22:54:21.474 INFO:teuthology.orchestra.run.smithi076.stdout:3902 punch from 0x83f728b to 0x83f904b, (0x1dc0 bytes) 2024-09-14T22:54:21.474 INFO:teuthology.orchestra.run.smithi076.stdout:3903 write 0x160b6a6 thru 0x1618d62 (0xd6bd bytes) 2024-09-14T22:54:21.483 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:3905 read 0x5de2afc thru 0x5de6bf2 (0x40f7 bytes) 2024-09-14T22:54:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:3907 read 0x565190 thru 0x5713e6 (0xc257 bytes) 2024-09-14T22:54:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:3908 punch from 0x75c5e03 to 0x75cc40f, (0x660c bytes) 2024-09-14T22:54:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:21.484 INFO:teuthology.orchestra.run.smithi076.stdout:3910 writesame 0x4b90457 thru 0x4b9a52e data_size 0x4(0xa0d8 bytes) 2024-09-14T22:54:21.491 INFO:teuthology.orchestra.run.smithi076.stdout:3913 read 0x1227e32 thru 0x12301c6 (0x8395 bytes) 2024-09-14T22:54:21.492 INFO:teuthology.orchestra.run.smithi076.stdout:3915 punch from 0x6876e48 to 0x6879a56, (0x2c0e bytes) 2024-09-14T22:54:21.492 INFO:teuthology.orchestra.run.smithi076.stdout:3917 punch from 0x5bcb972 to 0x5bd5385, (0x9a13 bytes) 2024-09-14T22:54:21.492 INFO:teuthology.orchestra.run.smithi076.stdout:3920 write 0x3db79c7 thru 0x3dc3207 (0xb841 bytes) 2024-09-14T22:54:21.496 INFO:teuthology.orchestra.run.smithi076.stdout:3921 clone 36 order 19 su 8192 sc 15 2024-09-14T22:54:21.503 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:21.503 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:21.509 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:21.510 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:21.519 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:21.529 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:21.530 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:21.530 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:21.528+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:21.546 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:21.562 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:21.579 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:21.579 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:21.596 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:21.596 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:21.613 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:21.613 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:21.629 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:21.628+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:21.630 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:21.630 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:21.635 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:21.635 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:21.635 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:21.636 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:21.645 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:21.730 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:21.729+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:21.788 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:21.789 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:21.789 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:21.789 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:21.789 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:21.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:21.831 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:21.830+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8992 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'23, mlcod=33'21)", 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.993301973999998, 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0035453559999999999, 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.902 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906374+0000", 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906376+0000", 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.42e-06 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906968+0000", 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00059252399999999998 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906990+0000", 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1498999999999999e-05 2024-09-14T22:54:21.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.906999+0000", 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2299999999999997e-06 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907058+0000", 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9505e-05 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.907082+0000", 2024-09-14T22:54:21.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3298999999999998e-05 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909874+0000", 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027926689999999998 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.909919+0000", 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5210000000000003e-05 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.906 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:8998 3.9 e34/24 3:937715c4:::rbd_data.113535cec067.00000000000000c9:head v 34'25, mlcod=33'21)", 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.991634040000001, 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003268272, 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908042+0000", 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908043+0000", 2024-09-14T22:54:21.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908054+0000", 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1654999999999999e-05 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908066+0000", 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1321000000000001e-05 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908077+0000", 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1781e-05 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908149+0000", 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1402000000000005e-05 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.908170+0000", 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0746000000000001e-05 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911259+0000", 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030889839999999999 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:17.911310+0000", 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1684999999999997e-05 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15153 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'58, mlcod=34'56)", 2024-09-14T22:54:21.911 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.189459622000001, 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043627650000000002, 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710216+0000", 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710218+0000", 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.291e-06 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710957+0000", 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00073905599999999998 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710977+0000", 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0506999999999999e-05 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.710986+0000", 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0860000000000008e-06 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711048+0000", 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1130999999999995e-05 2024-09-14T22:54:21.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711072+0000", 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4167000000000001e-05 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714533+0000", 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034611070000000002 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714579+0000", 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6419999999999999e-05 2024-09-14T22:54:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:15154 3.4 e34/24 3:23622c75:::rbd_data.113535cec067.00000000000001ac:head v 34'60, mlcod=34'56)", 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.188108866999997, 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00342618, 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.916 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711567+0000", 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711568+0000", 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0900000000000002e-07 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711574+0000", 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5119999999999997e-06 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711583+0000", 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7579999999999998e-06 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711589+0000", 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9619999999999998e-06 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711692+0000", 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000103237 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.711719+0000", 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.7124999999999999e-05 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714953+0000", 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032341240000000001 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:18.714993+0000", 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9953000000000002e-05 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.691154628, 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:54:21.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.923 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:78175 3.4 e41/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 41'105, mlcod=40'102)", 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 53.933875673999999, 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0049595489999999997, 2024-09-14T22:54:21.925 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965800+0000", 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965801+0000", 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.35e-07 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965834+0000", 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3683999999999998e-05 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965846+0000", 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1620999999999999e-05 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965855+0000", 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1130000000000007e-06 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965939+0000", 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4029000000000005e-05 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.965963+0000", 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3529000000000001e-05 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970705+0000", 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.004742853 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.970760+0000", 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4484999999999998e-05 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.929 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:116229 3.1e e48/44 3:7a496d5d:::rbd_object_map.1135d503d0a:head v 48'99, mlcod=48'97)", 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.595583773000001, 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0034125309999999999, 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304092+0000", 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304095+0000", 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3029999999999998e-06 2024-09-14T22:54:21.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304126+0000", 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1374999999999997e-05 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304140+0000", 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4518000000000001e-05 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304150+0000", 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304213+0000", 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3366000000000003e-05 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.304237+0000", 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4379000000000002e-05 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307471+0000", 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0032338829999999999 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.307505+0000", 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3552e-05 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:54:21.934 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.567492854000001, 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:21.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:54:21.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:133036 3.4 e54/24 3:22b239af:::rbd_data.1135a4365f9c.0000000000000001:head v 54'137, mlcod=53'135)", 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 39.629792905000002, 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002821278, 2024-09-14T22:54:21.939 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.269883+0000", 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7100000000000001e-07 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270380+0000", 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00049641500000000005 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270393+0000", 2024-09-14T22:54:21.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3088000000000001e-05 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270402+0000", 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6199999999999994e-06 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270470+0000", 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7459999999999994e-05 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.270498+0000", 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8214000000000001e-05 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272657+0000", 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021591499999999999 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:42.272704+0000", 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7160000000000002e-05 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.943 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.635725383, 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:21.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:54:21.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:142014 3.1e e65/44 3:7ae60eba:::rbd_data.1135a7a7c51e.0000000000000000:head v 65'220, mlcod=65'217)", 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.709722097, 2024-09-14T22:54:21.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0042481029999999996, 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189954+0000", 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.189955+0000", 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0979999999999999e-06 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191197+0000", 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0012418710000000001 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191217+0000", 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.978e-05 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191226+0000", 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6500000000000008e-06 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191288+0000", 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1227000000000002e-05 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.191312+0000", 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4088999999999999e-05 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.194153+0000", 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002841437 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:54.194202+0000", 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8950999999999999e-05 2024-09-14T22:54:21.952 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:155203 3.4 3:23c245a1:::rbd_data.1135d0236bc5.000000000000001b:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:04.172813+0000", 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 17.726862995000001, 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027271249999999999, 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:21.953 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 155203 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172813+0000", 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172813+0000", 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172815+0000", 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4720000000000001e-06 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172822+0000", 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4309999999999997e-06 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172836+0000", 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4105e-05 2024-09-14T22:54:21.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172843+0000", 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9550000000000003e-06 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172900+0000", 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7469000000000002e-05 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.172907+0000", 2024-09-14T22:54:21.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6470000000000001e-06 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.174386+0000", 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001478739 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.174459+0000", 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3569e-05 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:04.175540+0000", 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010807379999999999 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:167397 3.9 e78/24 3:90d76597:::rbd_object_map.113519349ea7:head v 78'195, mlcod=78'192)", 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:09.407817+0000", 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 12.491858617, 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003855163, 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.958 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407817+0000", 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407817+0000", 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407818+0000", 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407822+0000", 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9890000000000003e-06 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407830+0000", 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6159999999999996e-06 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407835+0000", 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2399999999999998e-06 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407901+0000", 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5820000000000003e-05 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.407914+0000", 2024-09-14T22:54:21.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3393e-05 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.411634+0000", 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0037199400000000001 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:09.411673+0000", 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8645e-05 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172275 3.9 e82/24 3:91bc404c:::rbd_data.113598f99d1a.0000000000000008:head v 82'230, mlcod=81'225)", 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.171731+0000", 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 7.7279447860000001, 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0028425109999999998, 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171731+0000", 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171731+0000", 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171732+0000", 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5300000000000004e-07 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171742+0000", 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6390000000000004e-06 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171754+0000", 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228e-05 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171762+0000", 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.541e-06 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171815+0000", 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2176000000000003e-05 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.171839+0000", 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4751000000000001e-05 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.174535+0000", 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0026956929999999999 2024-09-14T22:54:21.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.174574+0000", 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8829999999999999e-05 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172286 3.1e e82/44 3:7ab3fec2:::rbd_data.113598f99d1a.0000000000000020:head v 82'288, mlcod=81'284)", 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.172802+0000", 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 7.7268739469999996, 2024-09-14T22:54:21.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0037798530000000001, 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172802+0000", 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172802+0000", 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172803+0000", 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6400000000000002e-07 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172809+0000", 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6379999999999998e-06 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172820+0000", 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0305999999999999e-05 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172824+0000", 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6720000000000003e-06 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172864+0000", 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9882000000000002e-05 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172878+0000", 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3637e-05 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.176541+0000", 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036630500000000002 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.176582+0000", 2024-09-14T22:54:21.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1004000000000002e-05 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172283 3.9 e82/24 3:9284f145:::rbd_data.113598f99d1a.000000000000001c:head v 82'232, mlcod=81'225)", 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.172843+0000", 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 7.7268326969999999, 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003802728, 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172843+0000", 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172843+0000", 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172844+0000", 2024-09-14T22:54:21.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172849+0000", 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2179999999999998e-06 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172859+0000", 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0079e-05 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172866+0000", 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6220000000000003e-06 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172908+0000", 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1766e-05 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.172924+0000", 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6251000000000001e-05 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.176577+0000", 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036533410000000001 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.176646+0000", 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8719000000000004e-05 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 7.7257791410000003, 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:21.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:54:21.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172306 3.4 3:201e3f56:::rbd_data.113598f99d1a.0000000000000067:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.179710+0000", 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 7.7199662589999996, 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0037384850000000002, 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.981 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172306 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179710+0000", 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179710+0000", 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179712+0000", 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2299999999999998e-06 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179718+0000", 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7049999999999996e-06 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179733+0000", 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4905999999999999e-05 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179745+0000", 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2310000000000001e-05 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179816+0000", 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1408999999999993e-05 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179898+0000", 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1323999999999996e-05 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.180052+0000", 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00015394599999999999 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.182195+0000", 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0021436099999999998 2024-09-14T22:54:21.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.183314+0000", 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001118747 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.183404+0000", 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9815000000000007e-05 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.183448+0000", 2024-09-14T22:54:21.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4483000000000001e-05 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172324 3.4 3:213525dd:::rbd_data.113598f99d1a.00000000000000b5:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 7.7129891310000005, 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051499340000000001, 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:21.987 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172324 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:21.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186688+0000", 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1829999999999999e-06 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186693+0000", 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4100000000000001e-06 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186705+0000", 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2169e-05 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186712+0000", 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9789999999999996e-06 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186758+0000", 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6687000000000002e-05 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186839+0000", 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-05 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186980+0000", 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000141569 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191467+0000", 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0044866840000000003 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191734+0000", 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026690299999999998 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191793+0000", 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9447000000000002e-05 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191837+0000", 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3433000000000002e-05 2024-09-14T22:54:21.992 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172329 3.4 3:23ce00f5:::rbd_data.113598f99d1a.00000000000000ea:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.189174+0000", 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 7.7105020920000005, 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051388140000000002, 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:21.993 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172329 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189174+0000", 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189174+0000", 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:21.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189175+0000", 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2399999999999997e-07 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189179+0000", 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.839e-06 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189189+0000", 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8900000000000002e-06 2024-09-14T22:54:21.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189194+0000", 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.305e-06 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189244+0000", 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0052999999999997e-05 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:21.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189326+0000", 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2003999999999996e-05 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.189494+0000", 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016826800000000001 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.194024+0000", 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0045300150000000001 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:21.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.194261+0000", 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023702799999999999 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.194290+0000", 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9159999999999999e-05 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.194313+0000", 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2427999999999999e-05 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.998 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.999 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:21.999 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:21.999 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:22.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:22.000 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:21.999+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:22.101 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:22.100+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:22.118 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:54:22.125 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T22:54:22.141 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T22:54:22.150 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:22.151 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:22.151 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:22.160 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:22.202 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:22.200+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:22.230 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone35 from 0xa66053e (overlap 0x4c7ef3) to 0x0 2024-09-14T22:54:22.302 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:22.301+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:22.319 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:22.319 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:22.319 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:22.320 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:22.320 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:22.329 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:22.366 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-14T22:54:22.376 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.376 INFO:teuthology.orchestra.run.smithi076.stdout:3924 read 0x3e2583c thru 0x3e2f276 (0x9a3b bytes) 2024-09-14T22:54:22.377 INFO:teuthology.orchestra.run.smithi076.stdout:3925 write 0x6c13638 thru 0x6c1d665 (0xa02e bytes) 2024-09-14T22:54:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:3926 read 0x4ec1127 thru 0x4eca192 (0x906c bytes) 2024-09-14T22:54:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:3927 read 0x721e00f thru 0x7228985 (0xa977 bytes) 2024-09-14T22:54:22.390 INFO:teuthology.orchestra.run.smithi076.stdout:3930 punch from 0x744d6aa to 0x7455735, (0x808b bytes) 2024-09-14T22:54:22.400 INFO:teuthology.orchestra.run.smithi076.stdout:3932 write 0xd413ede thru 0xd419cdb (0x5dfe bytes) 2024-09-14T22:54:22.403 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:22.402+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:22.409 INFO:teuthology.orchestra.run.smithi076.stdout:3934 punch from 0x96e8a12 to 0x96f015c, (0x774a bytes) 2024-09-14T22:54:22.410 INFO:teuthology.orchestra.run.smithi076.stdout:3935 read 0xfac27d thru 0xfada0b (0x178f bytes) 2024-09-14T22:54:22.410 INFO:teuthology.orchestra.run.smithi076.stdout:3936 write 0x9989723 thru 0x998b4a9 (0x1d87 bytes) 2024-09-14T22:54:22.423 INFO:teuthology.orchestra.run.smithi076.stdout:3937 trunc from 0xd419cdc to 0x617afbe 2024-09-14T22:54:22.440 INFO:teuthology.orchestra.run.smithi076.stdout:3938 punch from 0x2a6d69f to 0x2a70b7c, (0x34dd bytes) 2024-09-14T22:54:22.451 INFO:teuthology.orchestra.run.smithi076.stdout:3941 trunc from 0x617afbe to 0xed7373b 2024-09-14T22:54:22.457 INFO:teuthology.orchestra.run.smithi076.stdout:3943 writesame 0xb7f1c08 thru 0xb7f74a2 data_size 0x1(0x589b bytes) 2024-09-14T22:54:22.474 INFO:teuthology.orchestra.run.smithi076.stdout:3946 writesame 0xb322133 thru 0xb328b5f data_size 0xb(0x6a2d bytes) 2024-09-14T22:54:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.482 INFO:teuthology.orchestra.run.smithi076.stdout:3949 punch from 0xdf91905 to 0xdf93325, (0x1a20 bytes) 2024-09-14T22:54:22.484 INFO:teuthology.orchestra.run.smithi076.stdout:3950 trunc from 0xed7373b to 0xcc14bb6 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585351355999997, 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.492 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:54:22.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:54:22.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.496 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585330820000003, 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.497 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585300820999997, 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:54:22.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585269668000002, 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.506 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:54:22.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:54:22.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.510 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585247011999996, 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.511 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:22.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585226511000002, 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.515 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.518 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:22.519 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585206749999998, 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.521 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:54:22.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585185965000001, 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.524 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.525 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:54:22.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585153552999998, 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:54:22.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:54:22.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.531 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585132154, 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.534 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:54:22.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:54:22.536 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.537 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585070754999997, 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.538 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:54:22.540 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.541 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585030842000002, 2024-09-14T22:54:22.542 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.543 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:22.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:54:22.545 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:54:22.546 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.585001085999998, 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.547 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.548 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:54:22.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:54:22.550 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.584970851000001, 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:54:22.551 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.552 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.555 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.584946111999997, 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:22.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.557 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.559 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.584922681999998, 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:54:22.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.561 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:54:22.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.584826946, 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:54:22.564 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:22.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.567 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:54:22.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.584798024000001, 2024-09-14T22:54:22.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:54:22.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.584774699, 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:54:22.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.577 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.452076955999999, 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:22.578 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:22.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:54:22.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:54:22.581 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.582 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:22.583 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:22.583 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:22.583 INFO:teuthology.orchestra.run.smithi076.stdout:3951 writesame 0x23453f6 thru 0x2354c49 data_size 0x2e(0xf854 bytes) 2024-09-14T22:54:22.584 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:22.584 INFO:teuthology.orchestra.run.smithi076.stdout:3952 trunc from 0xcc14bb6 to 0x69f0b8c 2024-09-14T22:54:22.585 INFO:teuthology.orchestra.run.smithi076.stdout:3953 trunc from 0x69f0b8c to 0x15c9041 2024-09-14T22:54:22.585 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.585 INFO:teuthology.orchestra.run.smithi076.stdout:3955 read 0xb906f2 thru 0xb90bf8 (0x507 bytes) 2024-09-14T22:54:22.587 INFO:teuthology.orchestra.run.smithi076.stdout:3956 writesame 0xd02b400 thru 0xd02f44f data_size 0x38(0x4050 bytes) 2024-09-14T22:54:22.587 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:22.586+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:22.591 INFO:teuthology.orchestra.run.smithi076.stdout:3957 trunc from 0xd02f450 to 0x9573782 2024-09-14T22:54:22.597 INFO:teuthology.orchestra.run.smithi076.stdout:3958 write 0x244447a thru 0x2452631 (0xe1b8 bytes) 2024-09-14T22:54:22.608 INFO:teuthology.orchestra.run.smithi076.stdout:3960 write 0x2051113 thru 0x20593d2 (0x82c0 bytes) 2024-09-14T22:54:22.612 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.618 INFO:teuthology.orchestra.run.smithi076.stdout:3962 trunc from 0x9573782 to 0xe6fb9ed 2024-09-14T22:54:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:3965 writesame 0xebc64a1 thru 0xebce284 data_size 0xe(0x7de4 bytes) 2024-09-14T22:54:22.644 INFO:teuthology.orchestra.run.smithi076.stdout:3967 trunc from 0xebce285 to 0x22938f2 2024-09-14T22:54:22.651 INFO:teuthology.orchestra.run.smithi076.stdout:3969 read 0x18234fa thru 0x182a223 (0x6d2a bytes) 2024-09-14T22:54:22.661 INFO:teuthology.orchestra.run.smithi076.stdout:3970 write 0x4dd1e75 thru 0x4ddf469 (0xd5f5 bytes) 2024-09-14T22:54:22.680 INFO:teuthology.orchestra.run.smithi076.stdout:3971 write 0x916a9a8 thru 0x9174d95 (0xa3ee bytes) 2024-09-14T22:54:22.686 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:22.685+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:22.687 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.687 INFO:teuthology.orchestra.run.smithi076.stdout:3974 punch from 0x2e7d0d1 to 0x2e7e06a, (0xf99 bytes) 2024-09-14T22:54:22.695 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:22.695 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:22.696 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:22.696 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:3975 writesame 0xc9679d8 thru 0xc96a54d data_size 0x1(0x2b76 bytes) 2024-09-14T22:54:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:3976 punch from 0x50e11a8 to 0x50ef804, (0xe65c bytes) 2024-09-14T22:54:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:3977 read 0x2705888 thru 0x270d326 (0x7a9f bytes) 2024-09-14T22:54:22.698 INFO:teuthology.orchestra.run.smithi076.stdout:3978 read 0x2a5e348 thru 0x2a5f785 (0x143e bytes) 2024-09-14T22:54:22.701 INFO:teuthology.orchestra.run.smithi076.stdout:3979 trunc from 0xc96a54e to 0x9eae617 2024-09-14T22:54:22.705 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:22.707 INFO:teuthology.orchestra.run.smithi076.stdout:3980 write 0x56b630a thru 0x56c0c69 (0xa960 bytes) 2024-09-14T22:54:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:3982 read 0x284fc02 thru 0x2853867 (0x3c66 bytes) 2024-09-14T22:54:22.714 INFO:teuthology.orchestra.run.smithi076.stdout:3984 write 0x2d14183 thru 0x2d18fcb (0x4e49 bytes) 2024-09-14T22:54:22.718 INFO:teuthology.orchestra.run.smithi076.stdout:3985 write 0x7ce3216 thru 0x7ced8e5 (0xa6d0 bytes) 2024-09-14T22:54:22.725 INFO:teuthology.orchestra.run.smithi076.stdout:3986 write 0x269f536 thru 0x269feb3 (0x97e bytes) 2024-09-14T22:54:22.727 INFO:teuthology.orchestra.run.smithi076.stdout:3989 writesame 0x3ab93f2 thru 0x3abaca7 data_size 0x1(0x18b6 bytes) 2024-09-14T22:54:22.734 INFO:teuthology.orchestra.run.smithi076.stdout:3990 punch from 0x3676961 to 0x368498c, (0xe02b bytes) 2024-09-14T22:54:22.734 INFO:teuthology.orchestra.run.smithi076.stdout:3991 write 0x6cd229d thru 0x6cd70ff (0x4e63 bytes) 2024-09-14T22:54:22.742 INFO:teuthology.orchestra.run.smithi076.stdout:3992 write 0x8dec34d thru 0x8decb89 (0x83d bytes) 2024-09-14T22:54:22.757 INFO:teuthology.orchestra.run.smithi076.stdout:3993 writesame 0xeabfac4 thru 0xeacf723 data_size 0x30(0xfc60 bytes) 2024-09-14T22:54:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:3995 punch from 0x6491527 to 0x649b3af, (0x9e88 bytes) 2024-09-14T22:54:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:3997 read 0x13934bf thru 0x13958ed (0x242f bytes) 2024-09-14T22:54:22.758 INFO:teuthology.orchestra.run.smithi076.stdout:3998 writesame 0x8b3297c thru 0x8b3c84e data_size 0x1(0x9ed3 bytes) 2024-09-14T22:54:22.787 INFO:teuthology.orchestra.run.smithi076.stdout:3999 trunc from 0xeacf724 to 0xda5d310 2024-09-14T22:54:22.787 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:22.786+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:22.795 INFO:teuthology.orchestra.run.smithi076.stdout:4000 write 0x57221c5 thru 0x572862d (0x6469 bytes) 2024-09-14T22:54:22.802 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.802 INFO:teuthology.orchestra.run.smithi076.stdout:4004 read 0xb655386 thru 0xb6624b0 (0xd12b bytes) 2024-09-14T22:54:22.802 INFO:teuthology.orchestra.run.smithi076.stdout:4005 write 0xd12e5a9 thru 0xd130434 (0x1e8c bytes) 2024-09-14T22:54:22.805 INFO:teuthology.orchestra.run.smithi076.stdout:4006 read 0xa66424c thru 0xa66ee16 (0xabcb bytes) 2024-09-14T22:54:22.805 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:22.821 INFO:teuthology.orchestra.run.smithi076.stdout:4008 trunc from 0xda5d310 to 0xe5f561 2024-09-14T22:54:22.861 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:22.861 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:22.861 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:22.861 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:22.861 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:22.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:22.879 INFO:teuthology.orchestra.run.smithi076.stdout:4009 writesame 0x4fff0d7 thru 0x500e99b data_size 0x2f(0xf8c5 bytes) 2024-09-14T22:54:22.887 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:22.886+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:4010 write 0x809c000 thru 0x809cefc (0xefd bytes) 2024-09-14T22:54:22.892 INFO:teuthology.orchestra.run.smithi076.stdout:4011 read 0x38967dd thru 0x3899526 (0x2d4a bytes) 2024-09-14T22:54:22.899 INFO:teuthology.orchestra.run.smithi076.stdout:4012 trunc from 0x809cefd to 0xdfc44e4 2024-09-14T22:54:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:4013 punch from 0xd0f631b to 0xd10411b, (0xde00 bytes) 2024-09-14T22:54:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:4014 read 0xb1a5ff0 thru 0xb1b527e (0xf28f bytes) 2024-09-14T22:54:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:4015 punch from 0xc15c3e4 to 0xc15c58c, (0x1a8 bytes) 2024-09-14T22:54:22.906 INFO:teuthology.orchestra.run.smithi076.stdout:4016 writesame 0x8502d7e thru 0x8510907 data_size 0x72(0xdb8a bytes) 2024-09-14T22:54:22.924 INFO:teuthology.orchestra.run.smithi076.stdout:4017 writesame 0xebc5b98 thru 0xebd056f data_size 0x14(0xa9d8 bytes) 2024-09-14T22:54:22.924 INFO:teuthology.orchestra.run.smithi076.stdout:4020 write 0x5425b3e thru 0x5434392 (0xe855 bytes) 2024-09-14T22:54:22.946 INFO:teuthology.orchestra.run.smithi076.stdout:4022 trunc from 0xebd0570 to 0x4e44052 2024-09-14T22:54:22.959 INFO:teuthology.orchestra.run.smithi076.stdout:4024 read 0x907f28 thru 0x91722a (0xf303 bytes) 2024-09-14T22:54:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:4026 punch from 0x132d3fe to 0x133cb24, (0xf726 bytes) 2024-09-14T22:54:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:4027 read 0xb57883 thru 0xb5aa6f (0x31ed bytes) 2024-09-14T22:54:22.960 INFO:teuthology.orchestra.run.smithi076.stdout:4028 read 0x264a04a thru 0x265532e (0xb2e5 bytes) 2024-09-14T22:54:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:4029 write 0xd6a70d3 thru 0xd6b0f2a (0x9e58 bytes) 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.927289315000003, 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:22.984 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:54:22.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:22.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:22.988 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.927094222999997, 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:22.989 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:54:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:22.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.926941433000003, 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:22.994 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:54:22.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:22.998 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.926736411, 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:54:22.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:23.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.926465745000002, 2024-09-14T22:54:23.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:54:23.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:54:23.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:23.008 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.926203585000003, 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:23.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:54:23.012 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.926044732999998, 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.013 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.925857937000004, 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.925655691999999, 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.022 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:54:23.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.925447333999998, 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:54:23.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:54:23.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.925133617, 2024-09-14T22:54:23.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:54:23.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.924927855, 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.036 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:54:23.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:54:23.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.924758979000003, 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:54:23.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:54:23.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:54:23.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.924488285000002, 2024-09-14T22:54:23.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:54:23.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:54:23.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.924339009999997, 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.054 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.924097768000003, 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:54:23.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:54:23.058 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.615407736000002, 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.059 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:54:23.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:54:23.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.064 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153281 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'291, mlcod=72'289)", 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.545898273999999, 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060467871999999999, 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435411+0000", 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5589999999999999e-06 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435419+0000", 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.909e-06 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435423+0000", 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5140000000000003e-06 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435458+0000", 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965000000000002e-05 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435470+0000", 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1559999999999999e-05 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495827+0000", 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060356819999999999 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495874+0000", 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7003999999999999e-05 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.069 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153282 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'292, mlcod=72'289)", 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.545266558000002, 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060246112999999997, 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436038+0000", 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436042+0000", 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8920000000000002e-06 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495165+0000", 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059122945000000003 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495179+0000", 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4185000000000001e-05 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495346+0000", 2024-09-14T22:54:23.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016677400000000001 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495373+0000", 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7172e-05 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496245+0000", 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00087206100000000004 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496284+0000", 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8525000000000001e-05 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:187784 3.2 3:45bacf70:::rbd_data.11354c7c4fd6.0000000000000028:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.150105838, 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.91484284000000005, 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.074 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 187784 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831199+0000", 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2200000000000002e-07 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831203+0000", 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1019999999999999e-06 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831213+0000", 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0087e-05 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831218+0000", 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2299999999999999e-06 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831275+0000", 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6200999999999999e-05 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831286+0000", 2024-09-14T22:54:23.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0838e-05 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.744712+0000", 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.91342615800000004 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.744890+0000", 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000178488 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 5", 2024-09-14T22:54:23.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745032+0000", 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014174500000000001 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745748+0000", 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00071614400000000005 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745984+0000", 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023544500000000001 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.746015+0000", 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1683000000000001e-05 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.746041+0000", 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5797000000000001e-05 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.080 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:23.082 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:4030 read 0x219c888 thru 0x21a3250 (0x69c9 bytes) 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:4031 write 0x7dee2eb thru 0x7df626e (0x7f84 bytes) 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:4032 trunc from 0xd6b0f2b to 0x10d8d06 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:4033 punch from 0x69addd to 0x69f331, (0x4554 bytes) 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:4034 writesame 0xb7976b7 thru 0xb7a6f9d data_size 0x1(0xf8e7 bytes) 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:4035 punch from 0x34470ea to 0x344ad5e, (0x3c74 bytes) 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:4037 read 0x2744c48 thru 0x27510a1 (0xc45a bytes) 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:4038 punch from 0x700d4e0 to 0x700e812, (0x1332 bytes) 2024-09-14T22:54:23.084 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:4040 read 0xab859bb thru 0xab85a62 (0xa8 bytes) 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:4042 trunc from 0xb7a6f9e to 0x87dbf71 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:4043 writesame 0x17812ea thru 0x17859f9 data_size 0x18(0x4710 bytes) 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:4044 write 0x52438a5 thru 0x5251eaa (0xe606 bytes) 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:4045 read 0x3826066 thru 0x382dd7d (0x7d18 bytes) 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:4046 writesame 0xa3b161d thru 0xa3bfc1f data_size 0x35(0xe603 bytes) 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:4047 read 0x17da335 thru 0x17e316f (0x8e3b bytes) 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:4051 write 0x7b6281f thru 0x7b6e96a (0xc14c bytes) 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:23.085 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:23.086 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:23.080+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:23.086 INFO:teuthology.orchestra.run.smithi076.stdout:4055 trunc from 0xa3bfc20 to 0xde0b02 2024-09-14T22:54:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:4058 writesame 0x4ba0f0d thru 0x4ba4330 data_size 0x47(0x3424 bytes) 2024-09-14T22:54:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:4059 punch from 0xb4ce42 to 0xb5c879, (0xfa37 bytes) 2024-09-14T22:54:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:4060 read 0x2711a1e thru 0x2719f95 (0x8578 bytes) 2024-09-14T22:54:23.112 INFO:teuthology.orchestra.run.smithi076.stdout:4061 writesame 0x96e173 thru 0x978710 data_size 0x3a(0xa59e bytes) 2024-09-14T22:54:23.114 INFO:teuthology.orchestra.run.smithi076.stdout:4062 write 0x31efe53 thru 0x31fb1ca (0xb378 bytes) 2024-09-14T22:54:23.132 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T22:54:23.147 INFO:teuthology.orchestra.run.smithi076.stdout:4063 write 0xa015634 thru 0xa019cbb (0x4688 bytes) 2024-09-14T22:54:23.148 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T22:54:23.164 INFO:teuthology.orchestra.run.smithi076.stdout:4064 write 0xd95ab81 thru 0xd96a5fe (0xfa7e bytes) 2024-09-14T22:54:23.178 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:23.178 INFO:teuthology.orchestra.run.smithi076.stdout:4066 write 0x6ea563d thru 0x6eac0a2 (0x6a66 bytes) 2024-09-14T22:54:23.182 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:23.181+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:23.190 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:23.190 INFO:teuthology.orchestra.run.smithi076.stdout:4069 punch from 0xcc24a92 to 0xcc2e450, (0x99be bytes) 2024-09-14T22:54:23.197 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:23.197 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:23.197 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:23.197 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:23.203 INFO:teuthology.orchestra.run.smithi076.stdout:4070 trunc from 0xd96a5ff to 0x3435141 2024-09-14T22:54:23.206 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:4071 read 0x15a0a9a thru 0x15a9d40 (0x92a7 bytes) 2024-09-14T22:54:23.219 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:23.235 INFO:teuthology.orchestra.run.smithi076.stdout:4073 write 0xbe8fbce thru 0xbe90592 (0x9c5 bytes) 2024-09-14T22:54:23.239 INFO:teuthology.orchestra.run.smithi076.stdout:4074 write 0x939b063 thru 0x939df99 (0x2f37 bytes) 2024-09-14T22:54:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:4075 read 0x9ac14c6 thru 0x9acfc6f (0xe7aa bytes) 2024-09-14T22:54:23.243 INFO:teuthology.orchestra.run.smithi076.stdout:4076 writesame 0x34779e6 thru 0x3481a1f data_size 0x1(0xa03a bytes) 2024-09-14T22:54:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:4077 punch from 0x1bdc96e to 0x1be9335, (0xc9c7 bytes) 2024-09-14T22:54:23.268 INFO:teuthology.orchestra.run.smithi076.stdout:4078 writesame 0x7eb5d55 thru 0x7ebba0c data_size 0xc(0x5cb8 bytes) 2024-09-14T22:54:23.274 INFO:teuthology.orchestra.run.smithi076.stdout:4079 read 0xb999872 thru 0xb99a7d6 (0xf65 bytes) 2024-09-14T22:54:23.274 INFO:teuthology.orchestra.run.smithi076.stdout:4081 write 0x47fdac thru 0x48ae5e (0xb0b3 bytes) 2024-09-14T22:54:23.283 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:23.282+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:23.306 INFO:teuthology.orchestra.run.smithi076.stdout:4082 writesame 0xd61fca5 thru 0xd629f6e data_size 0x2(0xa2ca bytes) 2024-09-14T22:54:23.307 INFO:teuthology.orchestra.run.smithi076.stdout:4083 writesame 0x537ad84 thru 0x5389572 data_size 0x5f(0xe7ef bytes) 2024-09-14T22:54:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:4085 punch from 0x28eb7da to 0x28f8ea9, (0xd6cf bytes) 2024-09-14T22:54:23.316 INFO:teuthology.orchestra.run.smithi076.stdout:4087 read 0x2c15e6 thru 0x2c2d16 (0x1731 bytes) 2024-09-14T22:54:23.317 INFO:teuthology.orchestra.run.smithi076.stdout:4089 writesame 0x4b504e8 thru 0x4b55bdf data_size 0x2e(0x56f8 bytes) 2024-09-14T22:54:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:4090 punch from 0x44c796c to 0x44cd5c6, (0x5c5a bytes) 2024-09-14T22:54:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:4091 punch from 0xb54d7fe to 0xb553e2b, (0x662d bytes) 2024-09-14T22:54:23.321 INFO:teuthology.orchestra.run.smithi076.stdout:4092 write 0xaf18a28 thru 0xaf20960 (0x7f39 bytes) 2024-09-14T22:54:23.323 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:23.323 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:23.323 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:23.323 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:23.323 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:23.331 INFO:teuthology.orchestra.run.smithi076.stdout:4093 clone 37 order 24 su 524288 sc 6 2024-09-14T22:54:23.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:23.385 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:23.383+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:23.436 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:23.436 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:23.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:23.436 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:23.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.436 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.538209516000002, 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:54:23.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:54:23.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:54:23.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:54:23.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:54:23.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127349 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4120576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.912519959999997, 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053205084, 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.444 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127349 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:23.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7000000000000005e-07 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521644+0000", 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521715+0000", 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0736999999999997e-05 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521725+0000", 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0237999999999999e-05 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574705+0000", 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052979928000000003 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574768+0000", 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2618000000000003e-05 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574846+0000", 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8143e-05 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.448 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.912415396999997, 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:54:23.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.912327404999999, 2024-09-14T22:54:23.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:23.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:54:23.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.912093487999996, 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.458 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:54:23.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:54:23.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.911912592999997, 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:23.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:54:23.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.911613090000003, 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:54:23.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:54:23.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127361 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.911292944000003, 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053260788000000003, 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127361 2024-09-14T22:54:23.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522869+0000", 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522872+0000", 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.066e-06 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522879+0000", 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8110000000000003e-06 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522885+0000", 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.003e-06 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575959+0000", 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073515000000002 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576037+0000", 2024-09-14T22:54:23.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7991999999999995e-05 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576129+0000", 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1772000000000002e-05 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:23.477 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.911266879999999, 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.910903152000003, 2024-09-14T22:54:23.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:54:23.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.910313535, 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:54:23.487 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:23.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:54:23.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.910111297999997, 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:54:23.492 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:54:23.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.909921486999998, 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:54:23.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.909609484000001, 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.909366996999999, 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.507 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:54:23.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:23.511 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.909208034999999, 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.512 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.908573765, 2024-09-14T22:54:23.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:54:23.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.908359654999998, 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:54:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:23.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:54:23.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.908223184999997, 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.526 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:54:23.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.530 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.907970704999997, 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.531 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:23.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:54:23.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.535 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.536 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:23.537 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:23.537 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:23.536+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:23.638 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:23.636+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:23.641 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:23.681 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:23.681 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:23.681 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:23.681 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:23.689 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:23.739 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:23.737+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:23.761 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:23.761 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:23.778 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:23.778 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:23.794 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:23.794 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:23.810 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:23.810 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:23.826 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:23.826 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:23.833 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:23.833 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:23.833 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:23.839 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:23.838+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:23.841 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:23.940 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:23.938+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:23.986 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:23.986 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:23.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:23.986 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:23.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.986 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:23.986 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:23.986 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.902333306999999, 2024-09-14T22:54:23.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:54:23.987 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.987 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.987 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.987 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.987 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.987 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.987 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:54:23.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:23.991 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.902115410999997, 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:54:23.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.899431765000003, 2024-09-14T22:54:23.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:23.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:23.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:23.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.899273350000001, 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:54:24.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:54:24.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.005 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.899246083999998, 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:24.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.899038943000001, 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:54:24.010 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:54:24.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.899004679000001, 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.015 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:54:24.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:54:24.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.897372633000003, 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.897348020999999, 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.024 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.028 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.897213735000001, 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.032 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.896164026999998, 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:54:24.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.037 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.896061842000002, 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:54:24.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:54:24.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:54:24.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:24.042 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.896037937999999, 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:24.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:54:24.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.896004417999997, 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.047 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:54:24.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.895849771999998, 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.052 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:54:24.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.895808598000002, 2024-09-14T22:54:24.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.060 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.895780438000003, 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.061 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:54:24.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.895591047000003, 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:54:24.065 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 59.979501220000003, 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.070 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:54:24.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.074 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 59.979481389, 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:24.075 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:24.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:54:24.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:54:24.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:24.079 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:24.080 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:24.081 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:24.081 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:24.082 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:24.082 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:24.082 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:24.083 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:24.083 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:24.079+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:24.136 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:54:24.142 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T22:54:24.159 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T22:54:24.181 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:24.180+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:24.189 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:24.189 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:24.197 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:24.282 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:24.281+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:24.340 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:24.341 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:24.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:24.383 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:24.382+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:24.484 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:24.483+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:24.493 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:24.493 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:24.502 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:24.584 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:24.583+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:24.604 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:24.605 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:24.613 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:24.685 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:24.684+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:24.756 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:24.756 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:24.765 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:24.808 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:24.807+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:24.869 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:25.008 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:25.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:25.011 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:25.009+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:25.110 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:25.109+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:54:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T22:54:25.158 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:25.158 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:25.171 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:25.173 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T22:54:25.214 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:25.213+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:25.276 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:25.276 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:25.286 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:25.330 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:25.329+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:25.395 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:25.396 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:25.404 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:25.445 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:25.444+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:25.514 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:25.513+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:25.548 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:25.549 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:25.557 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:25.615 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:25.614+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:25.701 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:25.701 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:25.709 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:25.716 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:25.715+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:25.812 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:25.812 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:25.816 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:25.815+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:25.820 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:25.917 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:25.916+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:25.965 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:25.965 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:25.973 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:26.018 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:26.017+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:26.076 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:26.077 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:26.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:26.088 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:26.126 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:26.125+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:26.159 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T22:54:26.177 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T22:54:26.219 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:26.218+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:26.242 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:26.242 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:26.255 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:26.264 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:26.264 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:26.284 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:26.284 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:26.304 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:26.304 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:26.320 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:26.319+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:26.323 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:26.323 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:26.346 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:26.347 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:26.353 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone36 from 0xd629f6f (overlap 0xde0b02) to 0xc7d6b5 2024-09-14T22:54:26.370 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:26.370 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:26.384 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:26.384 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:26.384 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:26.384 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:26.390 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:26.390 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:26.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:26.415 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:26.415 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:26.421 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:26.420+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:26.502 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-14T22:54:26.503 INFO:teuthology.orchestra.run.smithi076.stdout:4094 read 0x9830c02 thru 0x983d2df (0xc6de bytes) 2024-09-14T22:54:26.504 INFO:teuthology.orchestra.run.smithi076.stdout:4096 writesame 0x837777e thru 0x837a2a2 data_size 0x5(0x2b25 bytes) 2024-09-14T22:54:26.515 INFO:teuthology.orchestra.run.smithi076.stdout:4101 trunc from 0xd629f6f to 0xd898289 2024-09-14T22:54:26.520 INFO:teuthology.orchestra.run.smithi076.stdout:4103 read 0x40ec074 thru 0x40f6df8 (0xad85 bytes) 2024-09-14T22:54:26.522 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:26.520+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:26.527 INFO:teuthology.orchestra.run.smithi076.stdout:4104 write 0xe7f3fc9 thru 0xe7fc62a (0x8662 bytes) 2024-09-14T22:54:26.545 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:26.546 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:26.546 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:26.546 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:26.546 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:26.550 INFO:teuthology.orchestra.run.smithi076.stdout:4105 trunc from 0xe7fc62b to 0x7bd4d4 2024-09-14T22:54:26.554 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:26.619 INFO:teuthology.orchestra.run.smithi076.stdout:4106 read 0xbe3f0 thru 0xc2974 (0x4585 bytes) 2024-09-14T22:54:26.622 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:26.621+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:4109 writesame 0xe38d60f thru 0xe38e1c5 data_size 0x1(0xbb7 bytes) 2024-09-14T22:54:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:4110 punch from 0xa23e69f to 0xa2488d9, (0xa23a bytes) 2024-09-14T22:54:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:4111 write 0x5d9a09e thru 0x5da9d91 (0xfcf4 bytes) 2024-09-14T22:54:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:4112 punch from 0x1574a9c to 0x1581eb3, (0xd417 bytes) 2024-09-14T22:54:26.644 INFO:teuthology.orchestra.run.smithi076.stdout:4116 read 0x70d716f thru 0x70ddbd6 (0x6a68 bytes) 2024-09-14T22:54:26.658 INFO:teuthology.orchestra.run.smithi076.stdout:4117 trunc from 0xe38e1c6 to 0x2701beb 2024-09-14T22:54:26.677 INFO:teuthology.orchestra.run.smithi076.stdout:4118 write 0x837d505 thru 0x8381336 (0x3e32 bytes) 2024-09-14T22:54:26.684 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:26.684 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.628397411000002, 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.685 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:26.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:54:26.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.628202319000003, 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:54:26.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:26.694 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.628049529000002, 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:54:26.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.627844506999999, 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:54:26.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:54:26.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.627573841, 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.704 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:54:26.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:54:26.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.627311681000002, 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:54:26.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.627152829000003, 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:26.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:54:26.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.626966033000002, 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.719 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:54:26.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.723 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.626763787999998, 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:54:26.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:54:26.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.626555430000003, 2024-09-14T22:54:26.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:26.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:54:26.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.626241712999999, 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.733 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:54:26.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.737 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.738 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.738 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:26.738 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.626035950999999, 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:54:26.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:54:26.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.625867075000002, 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:54:26.745 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.746 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.746 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.746 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:54:26.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.748 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:54:26.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.752 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.625596381000001, 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:54:26.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.625447106000003, 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:54:26.757 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:54:26.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:54:26.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.625205864000002, 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:54:26.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.766 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.316515832, 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:54:26.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:54:26.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153281 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'291, mlcod=72'289)", 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.247006370000001, 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060467871999999999, 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435411+0000", 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5589999999999999e-06 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435419+0000", 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.909e-06 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435423+0000", 2024-09-14T22:54:26.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5140000000000003e-06 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435458+0000", 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965000000000002e-05 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435470+0000", 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1559999999999999e-05 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495827+0000", 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060356819999999999 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495874+0000", 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7003999999999999e-05 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.777 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153282 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'292, mlcod=72'289)", 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.246374654, 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060246112999999997, 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436038+0000", 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436042+0000", 2024-09-14T22:54:26.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8920000000000002e-06 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495165+0000", 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059122945000000003 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495179+0000", 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4185000000000001e-05 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495346+0000", 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016677400000000001 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495373+0000", 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7172e-05 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496245+0000", 2024-09-14T22:54:26.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00087206100000000004 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496284+0000", 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8525000000000001e-05 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:187784 3.2 3:45bacf70:::rbd_data.11354c7c4fd6.0000000000000028:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.8512139340000004, 2024-09-14T22:54:26.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.91484284000000005, 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 187784 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831199+0000", 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2200000000000002e-07 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831203+0000", 2024-09-14T22:54:26.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1019999999999999e-06 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831213+0000", 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0087e-05 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831218+0000", 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2299999999999999e-06 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831275+0000", 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6200999999999999e-05 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831286+0000", 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0838e-05 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.744712+0000", 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.91342615800000004 2024-09-14T22:54:26.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.744890+0000", 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000178488 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 5", 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745032+0000", 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014174500000000001 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745748+0000", 2024-09-14T22:54:26.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00071614400000000005 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745984+0000", 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023544500000000001 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.746015+0000", 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1683000000000001e-05 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.746041+0000", 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5797000000000001e-05 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.789 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:26.791 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:26.792 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:26.789+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:4119 trunc from 0x8381337 to 0xb90b3bc 2024-09-14T22:54:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:4120 write 0x6327aa3 thru 0x632845e (0x9bc bytes) 2024-09-14T22:54:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:4123 punch from 0x792058e to 0x792a6ca, (0xa13c bytes) 2024-09-14T22:54:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:4124 read 0x7fa94e5 thru 0x7fad40a (0x3f26 bytes) 2024-09-14T22:54:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:4125 punch from 0xa2d76f5 to 0xa2de9b0, (0x72bb bytes) 2024-09-14T22:54:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:4126 write 0x1e6218 thru 0x1f5157 (0xef40 bytes) 2024-09-14T22:54:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:4127 write 0xe24b888 thru 0xe259c64 (0xe3dd bytes) 2024-09-14T22:54:26.794 INFO:teuthology.orchestra.run.smithi076.stdout:4129 write 0x7a132d3 thru 0x7a1a12d (0x6e5b bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4130 read 0x7d7805 thru 0x7dcd16 (0x5512 bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4132 write 0x4dd5dd7 thru 0x4de0cc6 (0xaef0 bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4134 read 0xa69ea6b thru 0xa6a3285 (0x481b bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4136 read 0xb8f3010 thru 0xb8fcb23 (0x9b14 bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4137 trunc from 0xe259c65 to 0x95c8950 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4139 punch from 0x2c76f0f to 0x2c85937, (0xea28 bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4142 punch from 0x1a751e to 0x1a9bd3, (0x26b5 bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4143 write 0xa7ee6ae thru 0xa7f2d94 (0x46e7 bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4144 read 0x4935ae6 thru 0x493c13f (0x665a bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4145 write 0x9dfa600 thru 0x9dfd9fd (0x33fe bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4146 write 0x5bd1c6e thru 0x5bd56cf (0x3a62 bytes) 2024-09-14T22:54:26.795 INFO:teuthology.orchestra.run.smithi076.stdout:4149 write 0xb236bb4 thru 0xb2469d1 (0xfe1e bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4150 read 0x1914e71 thru 0x19194b2 (0x4642 bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4152 trunc from 0xb2469d2 to 0x83883cc 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4153 write 0xbbe71f3 thru 0xbbe8494 (0x12a2 bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4154 read 0x7ae95c9 thru 0x7af7a8d (0xe4c5 bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4155 punch from 0x6fbfdc6 to 0x6fc2d2a, (0x2f64 bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4158 punch from 0xb323c8b to 0xb32a573, (0x68e8 bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4159 punch from 0x5ed0c97 to 0x5edfa21, (0xed8a bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4161 write 0xc3d6609 thru 0xc3dc9b2 (0x63aa bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4162 read 0xc22931a thru 0xc232219 (0x8f00 bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4163 trunc from 0xc3dc9b3 to 0xa0d1529 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4165 read 0x5d6f6f5 thru 0x5d7e3c2 (0xecce bytes) 2024-09-14T22:54:26.796 INFO:teuthology.orchestra.run.smithi076.stdout:4166 read 0x93b79da thru 0x93c041c (0x8a43 bytes) 2024-09-14T22:54:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:4167 writesame 0xe681c5a thru 0xe6910ca data_size 0x33(0xf471 bytes) 2024-09-14T22:54:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.802 INFO:teuthology.orchestra.run.smithi076.stdout:4169 writesame 0x36bed8f thru 0x36c80e3 data_size 0x1(0x9355 bytes) 2024-09-14T22:54:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:4170 punch from 0x33097eb to 0x33197e5, (0xfffa bytes) 2024-09-14T22:54:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:4171 punch from 0xa920382 to 0xa92a661, (0xa2df bytes) 2024-09-14T22:54:26.827 INFO:teuthology.orchestra.run.smithi076.stdout:4173 read 0x6db9a2c thru 0x6dc8bef (0xf1c4 bytes) 2024-09-14T22:54:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:4176 read 0xe0ce626 thru 0xe0d9190 (0xab6b bytes) 2024-09-14T22:54:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.828 INFO:teuthology.orchestra.run.smithi076.stdout:4178 read 0x21a83a9 thru 0x21b485c (0xc4b4 bytes) 2024-09-14T22:54:26.829 INFO:teuthology.orchestra.run.smithi076.stdout:4179 writesame 0x2a2dd8f thru 0x2a3b4aa data_size 0x1a(0xd71c bytes) 2024-09-14T22:54:26.831 INFO:teuthology.orchestra.run.smithi076.stdout:4180 read 0x1518790 thru 0x151fab2 (0x7323 bytes) 2024-09-14T22:54:26.832 INFO:teuthology.orchestra.run.smithi076.stdout:4181 writesame 0x8ef5b6e thru 0x8ef8ad0 data_size 0x1(0x2f63 bytes) 2024-09-14T22:54:26.840 INFO:teuthology.orchestra.run.smithi076.stdout:4183 write 0x92701e thru 0x930555 (0x9538 bytes) 2024-09-14T22:54:26.842 INFO:teuthology.orchestra.run.smithi076.stdout:4184 read 0x2565fbf thru 0x2574ca3 (0xece5 bytes) 2024-09-14T22:54:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:4185 read 0x9edb059 thru 0x9ee07ab (0x5753 bytes) 2024-09-14T22:54:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:4188 read 0xc7c25ea thru 0xc7ceeb4 (0xc8cb bytes) 2024-09-14T22:54:26.844 INFO:teuthology.orchestra.run.smithi076.stdout:4190 writesame 0x5d57242 thru 0x5d5bcf9 data_size 0xc(0x4ab8 bytes) 2024-09-14T22:54:26.847 INFO:teuthology.orchestra.run.smithi076.stdout:4191 trunc from 0xe6910cb to 0xce488f4 2024-09-14T22:54:26.854 INFO:teuthology.orchestra.run.smithi076.stdout:4193 write 0x8b9ea2b thru 0x8ba0766 (0x1d3c bytes) 2024-09-14T22:54:26.855 INFO:teuthology.orchestra.run.smithi076.stdout:4194 read 0xcb2ec16 thru 0xcb3d8bd (0xeca8 bytes) 2024-09-14T22:54:26.871 INFO:teuthology.orchestra.run.smithi076.stdout:4195 trunc from 0xce488f4 to 0x23d219 2024-09-14T22:54:26.891 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:26.890+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:4196 writesame 0x943555d thru 0x9444427 data_size 0x29(0xeecb bytes) 2024-09-14T22:54:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.901 INFO:teuthology.orchestra.run.smithi076.stdout:4199 write 0x74bd087 thru 0x74c39a0 (0x691a bytes) 2024-09-14T22:54:26.904 INFO:teuthology.orchestra.run.smithi076.stdout:4200 write 0x4871992 thru 0x487bf4e (0xa5bd bytes) 2024-09-14T22:54:26.905 INFO:teuthology.orchestra.run.smithi076.stdout:4201 punch from 0x2b19dfa to 0x2b2112a, (0x7330 bytes) 2024-09-14T22:54:26.905 INFO:teuthology.orchestra.run.smithi076.stdout:4202 read 0x912d6a5 thru 0x91310a4 (0x3a00 bytes) 2024-09-14T22:54:26.908 INFO:teuthology.orchestra.run.smithi076.stdout:4204 trunc from 0x9444428 to 0xb746a4a 2024-09-14T22:54:26.913 INFO:teuthology.orchestra.run.smithi076.stdout:4205 punch from 0x529961d to 0x5299bff, (0x5e2 bytes) 2024-09-14T22:54:26.913 INFO:teuthology.orchestra.run.smithi076.stdout:4206 read 0x34b9813 thru 0x34c4447 (0xac35 bytes) 2024-09-14T22:54:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:4209 punch from 0x8dbe068 to 0x8dc3918, (0x58b0 bytes) 2024-09-14T22:54:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:4211 read 0xae5a633 thru 0xae64690 (0xa05e bytes) 2024-09-14T22:54:26.914 INFO:teuthology.orchestra.run.smithi076.stdout:4212 write 0x4f3bb6f thru 0x4f4b4b6 (0xf948 bytes) 2024-09-14T22:54:26.915 INFO:teuthology.orchestra.run.smithi076.stdout:4213 write 0x3a94ded thru 0x3a9bac8 (0x6cdc bytes) 2024-09-14T22:54:26.917 INFO:teuthology.orchestra.run.smithi076.stdout:4216 read 0x2ba94c3 thru 0x2bb834d (0xee8b bytes) 2024-09-14T22:54:26.917 INFO:teuthology.orchestra.run.smithi076.stdout:4217 write 0x2348633 thru 0x23545db (0xbfa9 bytes) 2024-09-14T22:54:26.926 INFO:teuthology.orchestra.run.smithi076.stdout:4219 write 0xcdbd1fd thru 0xcdccfec (0xfdf0 bytes) 2024-09-14T22:54:26.929 INFO:teuthology.orchestra.run.smithi076.stdout:4220 punch from 0x42db393 to 0x42e4c0c, (0x9879 bytes) 2024-09-14T22:54:26.929 INFO:teuthology.orchestra.run.smithi076.stdout:4221 read 0x6bd7632 thru 0x6bd8163 (0xb32 bytes) 2024-09-14T22:54:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:4222 punch from 0x96cbc0b to 0x96cfe24, (0x4219 bytes) 2024-09-14T22:54:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:4224 read 0x26c42d6 thru 0x26c4730 (0x45b bytes) 2024-09-14T22:54:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:4225 read 0x30adea5 thru 0x30afb8f (0x1ceb bytes) 2024-09-14T22:54:26.930 INFO:teuthology.orchestra.run.smithi076.stdout:4229 write 0x5a349c5 thru 0x5a443f2 (0xfa2e bytes) 2024-09-14T22:54:26.932 INFO:teuthology.orchestra.run.smithi076.stdout:4231 write 0x93be15f thru 0x93be72c (0x5ce bytes) 2024-09-14T22:54:26.936 INFO:teuthology.orchestra.run.smithi076.stdout:4232 write 0x9a3e898 thru 0x9a4507e (0x67e7 bytes) 2024-09-14T22:54:26.939 INFO:teuthology.orchestra.run.smithi076.stdout:4233 write 0x50ab972 thru 0x50b79b7 (0xc046 bytes) 2024-09-14T22:54:26.942 INFO:teuthology.orchestra.run.smithi076.stdout:4234 write 0x696e55c thru 0x6973792 (0x5237 bytes) 2024-09-14T22:54:26.945 INFO:teuthology.orchestra.run.smithi076.stdout:4237 write 0x7b77b33 thru 0x7b7fd85 (0x8253 bytes) 2024-09-14T22:54:26.946 INFO:teuthology.orchestra.run.smithi076.stdout:4238 write 0x8931a3a thru 0x8932443 (0xa0a bytes) 2024-09-14T22:54:26.948 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:26.948 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:26.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.948 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:194422 3.1f e94/24 3:f8abcf6d:::rbd_data.113513bad187.000000000000000a:head v 94'507, mlcod=94'505)", 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:26.947461+0000", 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.00032871899999999999, 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00035076600000000002, 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:26.947461+0000", 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:26.947461+0000", 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:26.947462+0000", 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4300000000000001e-07 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:26.947471+0000", 2024-09-14T22:54:26.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6590000000000002e-06 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:26.947485+0000", 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3382e-05 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:26.947492+0000", 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0190000000000001e-06 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:26.947536+0000", 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4901000000000003e-05 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:26.947549+0000", 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2866e-05 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 1 2024-09-14T22:54:26.952 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:26.953 INFO:teuthology.orchestra.run.smithi076.stdout:4239 write 0x9316267 thru 0x931d8fa (0x7694 bytes) 2024-09-14T22:54:26.953 INFO:teuthology.orchestra.run.smithi076.stdout:4241 write 0x7b5e0ae thru 0x7b60f24 (0x2e77 bytes) 2024-09-14T22:54:26.958 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:26.960 INFO:teuthology.orchestra.run.smithi076.stdout:4243 write 0xecf8572 thru 0xecfa804 (0x2293 bytes) 2024-09-14T22:54:26.963 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.971 INFO:teuthology.orchestra.run.smithi076.stdout:4245 trunc from 0xecfa805 to 0x96f3dfc 2024-09-14T22:54:26.982 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:26.982 INFO:teuthology.orchestra.run.smithi076.stdout:4247 write 0x171965a thru 0x1725e98 (0xc83f bytes) 2024-09-14T22:54:26.983 INFO:teuthology.orchestra.run.smithi076.stdout:4248 writesame 0x2cfa3f9 thru 0x2d0511c data_size 0xe(0xad24 bytes) 2024-09-14T22:54:26.991 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:26.990+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:26.994 INFO:teuthology.orchestra.run.smithi076.stdout:4249 write 0xb394202 thru 0xb398554 (0x4353 bytes) 2024-09-14T22:54:26.996 INFO:teuthology.orchestra.run.smithi076.stdout:4253 read 0x7c90cf3 thru 0x7c97c52 (0x6f60 bytes) 2024-09-14T22:54:26.996 INFO:teuthology.orchestra.run.smithi076.stdout:4254 read 0xa50427d thru 0xa50f2cf (0xb053 bytes) 2024-09-14T22:54:27.007 INFO:teuthology.orchestra.run.smithi076.stdout:4255 trunc from 0xb398555 to 0x28c176d 2024-09-14T22:54:27.019 INFO:teuthology.orchestra.run.smithi076.stdout:4256 read 0x1435137 thru 0x143c2a7 (0x7171 bytes) 2024-09-14T22:54:27.019 INFO:teuthology.orchestra.run.smithi076.stdout:4257 write 0x18821ed thru 0x188534e (0x3162 bytes) 2024-09-14T22:54:27.023 INFO:teuthology.orchestra.run.smithi076.stdout:4258 trunc from 0x28c176d to 0x483b17b 2024-09-14T22:54:27.028 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.030 INFO:teuthology.orchestra.run.smithi076.stdout:4260 trunc from 0x483b17b to 0x5f32ba4 2024-09-14T22:54:27.035 INFO:teuthology.orchestra.run.smithi076.stdout:4261 read 0x2fb8897 thru 0x2fc83ba (0xfb24 bytes) 2024-09-14T22:54:27.035 INFO:teuthology.orchestra.run.smithi076.stdout:4262 read 0x3e9c62e thru 0x3eaafae (0xe981 bytes) 2024-09-14T22:54:27.041 INFO:teuthology.orchestra.run.smithi076.stdout:4263 writesame 0x65efaf7 thru 0x65f0aaf data_size 0x19(0xfb9 bytes) 2024-09-14T22:54:27.041 INFO:teuthology.orchestra.run.smithi076.stdout:4264 write 0x466920b thru 0x467408d (0xae83 bytes) 2024-09-14T22:54:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:4266 punch from 0x235901 to 0x239dba, (0x44b9 bytes) 2024-09-14T22:54:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:4267 punch from 0x2664fdb to 0x266d151, (0x8176 bytes) 2024-09-14T22:54:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:4268 read 0x43fd268 thru 0x4409ed1 (0xcc6a bytes) 2024-09-14T22:54:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.043 INFO:teuthology.orchestra.run.smithi076.stdout:4271 read 0x32886d1 thru 0x328e6f6 (0x6026 bytes) 2024-09-14T22:54:27.044 INFO:teuthology.orchestra.run.smithi076.stdout:4272 read 0x128ae48 thru 0x128b752 (0x90b bytes) 2024-09-14T22:54:27.044 INFO:teuthology.orchestra.run.smithi076.stdout:4273 write 0x2b52b6b thru 0x2b5474e (0x1be4 bytes) 2024-09-14T22:54:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:4276 read 0xbd6e43 thru 0xbe4fbc (0xe17a bytes) 2024-09-14T22:54:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.045 INFO:teuthology.orchestra.run.smithi076.stdout:4280 read 0x443876c thru 0x443fa07 (0x729c bytes) 2024-09-14T22:54:27.046 INFO:teuthology.orchestra.run.smithi076.stdout:4281 read 0x3c17137 thru 0x3c2307d (0xbf47 bytes) 2024-09-14T22:54:27.046 INFO:teuthology.orchestra.run.smithi076.stdout:4283 read 0x3219bef thru 0x32208c3 (0x6cd5 bytes) 2024-09-14T22:54:27.061 INFO:teuthology.orchestra.run.smithi076.stdout:4287 write 0xe5358b2 thru 0xe541e96 (0xc5e5 bytes) 2024-09-14T22:54:27.064 INFO:teuthology.orchestra.run.smithi076.stdout:4288 read 0x5577bb7 thru 0x5583ae0 (0xbf2a bytes) 2024-09-14T22:54:27.064 INFO:teuthology.orchestra.run.smithi076.stdout:4289 punch from 0xd00fa32 to 0xd018632, (0x8c00 bytes) 2024-09-14T22:54:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:4290 read 0x3891fcf thru 0x3897813 (0x5845 bytes) 2024-09-14T22:54:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:4291 writesame 0x75892ef thru 0x7591ab1 data_size 0x23(0x87c3 bytes) 2024-09-14T22:54:27.068 INFO:teuthology.orchestra.run.smithi076.stdout:4295 punch from 0x91fc0d to 0x92f168, (0xf55b bytes) 2024-09-14T22:54:27.072 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:27.072 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:27.073 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:27.076 INFO:teuthology.orchestra.run.smithi076.stdout:4296 trunc from 0xe541e97 to 0x78ec017 2024-09-14T22:54:27.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:27.093 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:27.091+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:27.096 INFO:teuthology.orchestra.run.smithi076.stdout:4297 trunc from 0x78ec017 to 0x383edb 2024-09-14T22:54:27.125 INFO:teuthology.orchestra.run.smithi076.stdout:4298 write 0xa4e26c9 thru 0xa4ea3c9 (0x7d01 bytes) 2024-09-14T22:54:27.128 INFO:teuthology.orchestra.run.smithi076.stdout:4301 write 0x112eebf thru 0x11371e6 (0x8328 bytes) 2024-09-14T22:54:27.130 INFO:teuthology.orchestra.run.smithi076.stdout:4302 write 0x145f310 thru 0x146b819 (0xc50a bytes) 2024-09-14T22:54:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:4303 read 0x260862 thru 0x267ccb (0x746a bytes) 2024-09-14T22:54:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:4304 punch from 0x46042a4 to 0x460c568, (0x82c4 bytes) 2024-09-14T22:54:27.132 INFO:teuthology.orchestra.run.smithi076.stdout:4305 read 0x9b8720 thru 0x9c2183 (0x9a64 bytes) 2024-09-14T22:54:27.133 INFO:teuthology.orchestra.run.smithi076.stdout:4306 read 0x465d173 thru 0x4661c20 (0x4aae bytes) 2024-09-14T22:54:27.133 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.133 INFO:teuthology.orchestra.run.smithi076.stdout:4309 write 0x1450a5e thru 0x1458af6 (0x8099 bytes) 2024-09-14T22:54:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:4311 read 0x610d87 thru 0x616e85 (0x60ff bytes) 2024-09-14T22:54:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:4316 punch from 0x469c9f8 to 0x46a0b51, (0x4159 bytes) 2024-09-14T22:54:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.136 INFO:teuthology.orchestra.run.smithi076.stdout:4319 writesame 0x2b32a87 thru 0x2b3dd82 data_size 0x91(0xb2fc bytes) 2024-09-14T22:54:27.144 INFO:teuthology.orchestra.run.smithi076.stdout:4320 write 0xbb9c51c thru 0xbba7e8a (0xb96f bytes) 2024-09-14T22:54:27.151 INFO:teuthology.orchestra.run.smithi076.stdout:4321 writesame 0xbc28c1b thru 0xbc2f23b data_size 0x2d(0x6621 bytes) 2024-09-14T22:54:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:4322 read 0x3f8e27a thru 0x3f94706 (0x648d bytes) 2024-09-14T22:54:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.152 INFO:teuthology.orchestra.run.smithi076.stdout:4329 trunc from 0xbc2f23c to 0xb61ea3e 2024-09-14T22:54:27.160 INFO:teuthology.orchestra.run.smithi076.stdout:4330 punch from 0x286167c to 0x2866e36, (0x57ba bytes) 2024-09-14T22:54:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:4334 punch from 0x5d73f59 to 0x5d74927, (0x9ce bytes) 2024-09-14T22:54:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:4335 read 0xac42dca thru 0xac471f5 (0x442c bytes) 2024-09-14T22:54:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:4336 punch from 0x4f7072 to 0x4f7915, (0x8a3 bytes) 2024-09-14T22:54:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:4337 read 0x470923d thru 0x470d3cb (0x418f bytes) 2024-09-14T22:54:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:4338 read 0x14a6dbe thru 0x14b5471 (0xe6b4 bytes) 2024-09-14T22:54:27.161 INFO:teuthology.orchestra.run.smithi076.stdout:4339 writesame 0x516d458 thru 0x5175130 data_size 0x1(0x7cd9 bytes) 2024-09-14T22:54:27.171 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T22:54:27.173 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:54:27.184 INFO:teuthology.orchestra.run.smithi076.stdout:4340 punch from 0x15c3462 to 0x15cf493, (0xc031 bytes) 2024-09-14T22:54:27.184 INFO:teuthology.orchestra.run.smithi076.stdout:4341 punch from 0x8511898 to 0x8511c63, (0x3cb bytes) 2024-09-14T22:54:27.185 INFO:teuthology.orchestra.run.smithi076.stdout:4342 punch from 0x33a4158 to 0x33aa29b, (0x6143 bytes) 2024-09-14T22:54:27.185 INFO:teuthology.orchestra.run.smithi076.stdout:4343 punch from 0x9fba894 to 0x9fbee7f, (0x45eb bytes) 2024-09-14T22:54:27.189 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T22:54:27.191 INFO:teuthology.orchestra.run.smithi076.stdout:4345 trunc from 0xb61ea3e to 0x5bd913c 2024-09-14T22:54:27.193 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:27.192+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:27.196 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:27.196 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.298415939999998, 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:54:27.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:54:27.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:54:27.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127349 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4120576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.204 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.672726384000001, 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053205084, 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127349 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521641+0000", 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7000000000000005e-07 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521644+0000", 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8499999999999998e-06 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521715+0000", 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0736999999999997e-05 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521725+0000", 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0237999999999999e-05 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574705+0000", 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052979928000000003 2024-09-14T22:54:27.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574768+0000", 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2618000000000003e-05 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574846+0000", 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8143e-05 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.672621821, 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.672533829000002, 2024-09-14T22:54:27.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:54:27.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.672299912, 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.219 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:54:27.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.223 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.672119017, 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:27.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:54:27.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.671819513999999, 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.229 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:54:27.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.233 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127361 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.671499367999999, 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053260788000000003, 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127361 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522868+0000", 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522869+0000", 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:27.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522872+0000", 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.066e-06 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522879+0000", 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8110000000000003e-06 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522885+0000", 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.003e-06 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575959+0000", 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053073515000000002 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576037+0000", 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7991999999999995e-05 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576129+0000", 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1772000000000002e-05 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.671473304000003, 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.238 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.671109575999999, 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.243 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:54:27.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:27.247 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.670519959000003, 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:27.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:54:27.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.670317722, 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.252 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:27.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:54:27.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:54:27.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:54:27.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.670127911000002, 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:54:27.258 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:54:27.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.669815907999997, 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:54:27.263 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:27.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:54:27.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:54:27.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.669573421000003, 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.268 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:54:27.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.669414459000002, 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:54:27.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:54:27.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.668780189000003, 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:54:27.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.668566079000001, 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:54:27.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:54:27.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:54:27.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.668429609, 2024-09-14T22:54:27.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:27.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:54:27.291 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.668177129, 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:27.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.296 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:27.299 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:27.299 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:27.296+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4346 read 0x4820d55 thru 0x48226c6 (0x1972 bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4347 write 0x21da64 thru 0x225b86 (0x8123 bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4348 write 0x96ae0f5 thru 0x96b3939 (0x5845 bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4350 write 0x8776440 thru 0x87788d1 (0x2492 bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4356 write 0xc3221cb thru 0xc328dd8 (0x6c0e bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4358 read 0xbf30c thru 0xca8b0 (0xb5a5 bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4360 read 0x21ba945 thru 0x21c95d2 (0xec8e bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4361 punch from 0x1f19897 to 0x1f222d4, (0x8a3d bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4362 punch from 0x3abc236 to 0x3ac0d24, (0x4aee bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4363 read 0x6f232b9 thru 0x6f299c7 (0x670f bytes) 2024-09-14T22:54:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:4365 punch from 0x806b0f6 to 0x806bc0b, (0xb15 bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4367 trunc from 0xc328dd9 to 0x891211f 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4369 read 0x1f36c2c thru 0x1f3e142 (0x7517 bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4371 write 0x3f0e7c4 thru 0x3f103e1 (0x1c1e bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4373 punch from 0x729d7e4 to 0x72aab98, (0xd3b4 bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4375 trunc from 0x891211f to 0x74b2fb0 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4378 punch from 0x2561b5a to 0x256fc26, (0xe0cc bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4379 trunc from 0x74b2fb0 to 0xd3dcf89 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4380 writesame 0xae9e4ba thru 0xaeab1e9 data_size 0x31(0xcd30 bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4381 writesame 0x2261b11 thru 0x226ff5c data_size 0x26(0xe44c bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4382 write 0x2219c96 thru 0x22284b1 (0xe81c bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4383 write 0xbc1d79d thru 0xbc28033 (0xa897 bytes) 2024-09-14T22:54:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:4385 writesame 0x21f50e2 thru 0x21f6b64 data_size 0x1(0x1a83 bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4386 read 0x67877ba thru 0x6791624 (0x9e6b bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4388 read 0x2254b77 thru 0x225ff26 (0xb3b0 bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4390 read 0xc66e6c4 thru 0xc67703f (0x897c bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4392 punch from 0x68ef40e to 0x68fe235, (0xee27 bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4394 read 0x334e1a4 thru 0x335d644 (0xf4a1 bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4395 writesame 0x84e9e47 thru 0x84f8e1b data_size 0x31(0xefd5 bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4396 punch from 0xb4461f7 to 0xb45112f, (0xaf38 bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4397 read 0x786ca6 thru 0x79526d (0xe5c8 bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4398 read 0x402aa7 thru 0x40ab48 (0x80a2 bytes) 2024-09-14T22:54:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:4400 read 0x915f4c4 thru 0x916a799 (0xb2d6 bytes) 2024-09-14T22:54:27.308 INFO:teuthology.orchestra.run.smithi076.stdout:4401 trunc from 0xd3dcf89 to 0x773d953 2024-09-14T22:54:27.316 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.316 INFO:teuthology.orchestra.run.smithi076.stdout:4403 writesame 0x40bb7a9 thru 0x40c822d data_size 0x1(0xca85 bytes) 2024-09-14T22:54:27.355 INFO:teuthology.orchestra.run.smithi076.stdout:4404 trunc from 0x773d953 to 0xe79fb16 2024-09-14T22:54:27.360 INFO:teuthology.orchestra.run.smithi076.stdout:4405 writesame 0x4081007 thru 0x408fbeb data_size 0x1(0xebe5 bytes) 2024-09-14T22:54:27.399 INFO:teuthology.orchestra.run.smithi076.stdout:4406 punch from 0xc57500f to 0xc58089e, (0xb88f bytes) 2024-09-14T22:54:27.399 INFO:teuthology.orchestra.run.smithi076.stdout:4408 punch from 0x20301bf to 0x2039093, (0x8ed4 bytes) 2024-09-14T22:54:27.399 INFO:teuthology.orchestra.run.smithi076.stdout:4409 trunc from 0xe79fb16 to 0xe4d09c3 2024-09-14T22:54:27.399 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:27.397+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:27.408 INFO:teuthology.orchestra.run.smithi076.stdout:4410 writesame 0x42718ce thru 0x4281499 data_size 0x6e(0xfbcc bytes) 2024-09-14T22:54:27.411 INFO:teuthology.orchestra.run.smithi076.stdout:4413 read 0x1bb9e58 thru 0x1bba93a (0xae3 bytes) 2024-09-14T22:54:27.411 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.414 INFO:teuthology.orchestra.run.smithi076.stdout:4417 trunc from 0xe4d09c3 to 0xc2a046a 2024-09-14T22:54:27.419 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.419 INFO:teuthology.orchestra.run.smithi076.stdout:4419 read 0x7b45f15 thru 0x7b4c013 (0x60ff bytes) 2024-09-14T22:54:27.420 INFO:teuthology.orchestra.run.smithi076.stdout:4420 punch from 0x719d586 to 0x71a8ce1, (0xb75b bytes) 2024-09-14T22:54:27.420 INFO:teuthology.orchestra.run.smithi076.stdout:4421 punch from 0x70f189c to 0x70f1c68, (0x3cc bytes) 2024-09-14T22:54:27.420 INFO:teuthology.orchestra.run.smithi076.stdout:4424 read 0x271aed1 thru 0x271e24e (0x337e bytes) 2024-09-14T22:54:27.429 INFO:teuthology.orchestra.run.smithi076.stdout:4425 writesame 0xed4723e thru 0xed4b468 data_size 0x1(0x422b bytes) 2024-09-14T22:54:27.429 INFO:teuthology.orchestra.run.smithi076.stdout:4426 read 0xe0ff8bb thru 0xe1068f7 (0x703d bytes) 2024-09-14T22:54:27.429 INFO:teuthology.orchestra.run.smithi076.stdout:4427 write 0x40495a0 thru 0x40556eb (0xc14c bytes) 2024-09-14T22:54:27.430 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.430 INFO:teuthology.orchestra.run.smithi076.stdout:4429 write 0xb2bae7e thru 0xb2c6118 (0xb29b bytes) 2024-09-14T22:54:27.433 INFO:teuthology.orchestra.run.smithi076.stdout:4431 punch from 0xabc7673 to 0xabcb672, (0x3fff bytes) 2024-09-14T22:54:27.433 INFO:teuthology.orchestra.run.smithi076.stdout:4432 punch from 0xcc4ecf6 to 0xcc52aa0, (0x3daa bytes) 2024-09-14T22:54:27.433 INFO:teuthology.orchestra.run.smithi076.stdout:4435 writesame 0x8a6f9d6 thru 0x8a78dbe data_size 0x1(0x93e9 bytes) 2024-09-14T22:54:27.444 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:27.444 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:27.444 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:27.444 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:27.453 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:27.454 INFO:teuthology.orchestra.run.smithi076.stdout:4436 write 0xcc4044a thru 0xcc4b1bf (0xad76 bytes) 2024-09-14T22:54:27.464 INFO:teuthology.orchestra.run.smithi076.stdout:4438 trunc from 0xed4b469 to 0x8b8fd23 2024-09-14T22:54:27.485 INFO:teuthology.orchestra.run.smithi076.stdout:4439 writesame 0xb1015e4 thru 0xb10bde7 data_size 0x2(0xa804 bytes) 2024-09-14T22:54:27.486 INFO:teuthology.orchestra.run.smithi076.stdout:4441 write 0xc90247 thru 0xc929de (0x2798 bytes) 2024-09-14T22:54:27.487 INFO:teuthology.orchestra.run.smithi076.stdout:4442 write 0x6954a33 thru 0x6959774 (0x4d42 bytes) 2024-09-14T22:54:27.489 INFO:teuthology.orchestra.run.smithi076.stdout:4443 punch from 0x45e0ca9 to 0x45ef034, (0xe38b bytes) 2024-09-14T22:54:27.489 INFO:teuthology.orchestra.run.smithi076.stdout:4445 read 0x1e760c0 thru 0x1e7d29e (0x71df bytes) 2024-09-14T22:54:27.489 INFO:teuthology.orchestra.run.smithi076.stdout:4446 writesame 0x8fe7e18 thru 0x8ff4098 data_size 0x1d(0xc281 bytes) 2024-09-14T22:54:27.505 INFO:teuthology.orchestra.run.smithi076.stdout:4447 trunc from 0xb10bde8 to 0xd90022 2024-09-14T22:54:27.525 INFO:teuthology.orchestra.run.smithi076.stdout:4448 writesame 0x2f75f3a thru 0x2f7d138 data_size 0xb(0x71ff bytes) 2024-09-14T22:54:27.527 INFO:teuthology.orchestra.run.smithi076.stdout:4449 punch from 0x565edb to 0x567b28, (0x1c4d bytes) 2024-09-14T22:54:27.527 INFO:teuthology.orchestra.run.smithi076.stdout:truncating to largest ever: 0xedc8a89 2024-09-14T22:54:27.537 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:27.536+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:27.541 INFO:teuthology.orchestra.run.smithi076.stdout:4451 trunc from 0x2f7d139 to 0xedc8a89 2024-09-14T22:54:27.546 INFO:teuthology.orchestra.run.smithi076.stdout:4453 write 0x6557cc0 thru 0x6563e98 (0xc1d9 bytes) 2024-09-14T22:54:27.549 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:27.549 INFO:teuthology.orchestra.run.smithi076.stdout:4455 writesame 0xdd1b8bb thru 0xdd2928e data_size 0x12(0xd9d4 bytes) 2024-09-14T22:54:27.554 INFO:teuthology.orchestra.run.smithi076.stdout:4457 read 0x6763ef4 thru 0x676ff6d (0xc07a bytes) 2024-09-14T22:54:27.554 INFO:teuthology.orchestra.run.smithi076.stdout:4458 read 0x2545b6b thru 0x254bfcf (0x6465 bytes) 2024-09-14T22:54:27.555 INFO:teuthology.orchestra.run.smithi076.stdout:4460 write 0x365e324 thru 0x3664093 (0x5d70 bytes) 2024-09-14T22:54:27.556 INFO:teuthology.orchestra.run.smithi076.stdout:4462 write 0x3d69c24 thru 0x3d6a10c (0x4e9 bytes) 2024-09-14T22:54:27.557 INFO:teuthology.orchestra.run.smithi076.stdout:4463 punch from 0x94ee622 to 0x94f1c7f, (0x365d bytes) 2024-09-14T22:54:27.557 INFO:teuthology.orchestra.run.smithi076.stdout:4465 write 0x479dfe3 thru 0x47a2900 (0x491e bytes) 2024-09-14T22:54:27.559 INFO:teuthology.orchestra.run.smithi076.stdout:4466 read 0xe117cd9 thru 0xe11a427 (0x274f bytes) 2024-09-14T22:54:27.559 INFO:teuthology.orchestra.run.smithi076.stdout:4468 write 0x36f7d5 thru 0x379828 (0xa054 bytes) 2024-09-14T22:54:27.568 INFO:teuthology.orchestra.run.smithi076.stdout:4469 trunc from 0xedc8a89 to 0x85df429 2024-09-14T22:54:27.590 INFO:teuthology.orchestra.run.smithi076.stdout:4470 write 0xd255712 thru 0xd25f181 (0x9a70 bytes) 2024-09-14T22:54:27.592 INFO:teuthology.orchestra.run.smithi076.stdout:4471 read 0x4f8f783 thru 0x4f9b914 (0xc192 bytes) 2024-09-14T22:54:27.593 INFO:teuthology.orchestra.run.smithi076.stdout:4473 write 0xc659139 thru 0xc65fa47 (0x690f bytes) 2024-09-14T22:54:27.600 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:27.599+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:27.601 INFO:teuthology.orchestra.run.smithi076.stdout:4474 trunc from 0xd25f182 to 0x7817c9c 2024-09-14T22:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:27.602 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:27.602 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:27.610 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:27.611 INFO:teuthology.orchestra.run.smithi076.stdout:4475 read 0x23fed9e thru 0x2403ca1 (0x4f04 bytes) 2024-09-14T22:54:27.623 INFO:teuthology.orchestra.run.smithi076.stdout:4476 write 0xc4ff7a0 thru 0xc50e7d2 (0xf033 bytes) 2024-09-14T22:54:27.702 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:27.700+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:27.756 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:27.756 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:27.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.672592984000005, 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:54:27.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:54:27.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:27.761 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.672375087999995, 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:54:27.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:54:27.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.669691442000001, 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.766 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.770 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.669533027, 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:27.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:54:27.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.669505760999996, 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:54:27.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:54:27.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.780 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.669298620000006, 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:54:27.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.669264355999999, 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.785 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:54:27.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.667632310000002, 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.790 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:27.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:54:27.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:54:27.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.795 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.667607697999998, 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.796 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.667473412000007, 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:54:27.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:54:27.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.666423703999996, 2024-09-14T22:54:27.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:54:27.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:54:27.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.666321518999993, 2024-09-14T22:54:27.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:27.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.666297615000005, 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.817 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:27.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:54:27.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.666264095000002, 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:54:27.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:54:27.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.666109449000004, 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:54:27.826 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.827 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.827 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.827 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.827 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.827 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.827 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:54:27.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.827 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:54:27.831 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.666068275000001, 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.832 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.836 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.666040115000001, 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:54:27.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:54:27.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:27.841 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.665850723999995, 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.749760897000002, 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.846 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:54:27.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.850 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.749741065999999, 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:27.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:54:27.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:27.855 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:27.857 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:27.857 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:27.855+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:27.958 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:27.956+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:28.001 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:28.001 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:28.001 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:28.001 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:28.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:28.059 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:28.058+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:28.113 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:28.113 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:28.113 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:28.121 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:28.160 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:28.159+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:28.173 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T22:54:28.173 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:54:28.180 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.180 INFO:teuthology.orchestra.run.smithi076.stdout:4479 write 0xb57acfd thru 0xb585ff4 (0xb2f8 bytes) 2024-09-14T22:54:28.185 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.185 INFO:teuthology.orchestra.run.smithi076.stdout:4481 writesame 0x1aa6562 thru 0x1ab4781 data_size 0x86(0xe220 bytes) 2024-09-14T22:54:28.187 INFO:teuthology.orchestra.run.smithi076.stdout:4482 punch from 0xb5fd3a5 to 0xb609c57, (0xc8b2 bytes) 2024-09-14T22:54:28.187 INFO:teuthology.orchestra.run.smithi076.stdout:4483 write 0x3330e30 thru 0x333b397 (0xa568 bytes) 2024-09-14T22:54:28.189 INFO:teuthology.orchestra.run.smithi076.stdout:4484 writesame 0x31deb4a thru 0x31ea456 data_size 0x1(0xb90d bytes) 2024-09-14T22:54:28.192 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T22:54:28.193 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:54:28.220 INFO:teuthology.orchestra.run.smithi076.stdout:4485 read 0x58d26ac thru 0x58dc5ba (0x9f0f bytes) 2024-09-14T22:54:28.222 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.222 INFO:teuthology.orchestra.run.smithi076.stdout:4488 writesame 0x58f9ebe thru 0x59034e3 data_size 0x1(0x9626 bytes) 2024-09-14T22:54:28.230 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:28.230 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:28.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:28.230 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:28.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 95.720678199000005, 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:54:28.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:54:28.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:54:28.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.237 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.856020344000001, 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.238 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.242 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855914407999997, 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:54:28.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:54:28.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:28.247 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855839707000001, 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:54:28.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855754226000002, 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.252 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:54:28.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:54:28.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855607839999998, 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:28.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.261 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855557840000003, 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:54:28.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855394676000003, 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.266 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:54:28.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.270 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855356366000002, 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:54:28.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:54:28.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:54:28.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855222697000002, 2024-09-14T22:54:28.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:28.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:54:28.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:54:28.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:54:28.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.280 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855142129999997, 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.281 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:54:28.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.282 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:28.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:54:28.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.286 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.855045193000002, 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.854993638000003, 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.292 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:54:28.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.296 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.851869008999998, 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:54:28.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.851737395999997, 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.301 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:54:28.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.305 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.851450984000003, 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:28.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:28.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:54:28.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.793414729999999, 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.311 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:54:28.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:28.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.316 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.793374902, 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.317 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:54:28.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:54:28.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.793358887, 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:54:28.323 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:54:28.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:54:28.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:28.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:28.329 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.744230532, 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:54:28.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:28.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:28.335 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:28.336 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:28.338 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:28.335+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:28.338 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:28.339 INFO:teuthology.orchestra.run.smithi076.stdout:4490 trunc from 0xc50e7d3 to 0xb18856 2024-09-14T22:54:28.339 INFO:teuthology.orchestra.run.smithi076.stdout:4491 punch from 0x4e22d1 to 0x4e3af3, (0x1822 bytes) 2024-09-14T22:54:28.339 INFO:teuthology.orchestra.run.smithi076.stdout:4492 write 0x53417fd thru 0x53460e7 (0x48eb bytes) 2024-09-14T22:54:28.339 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.339 INFO:teuthology.orchestra.run.smithi076.stdout:4494 punch from 0x390c115 to 0x390f0a1, (0x2f8c bytes) 2024-09-14T22:54:28.339 INFO:teuthology.orchestra.run.smithi076.stdout:4495 read 0x5f7a72 thru 0x60078d (0x8d1c bytes) 2024-09-14T22:54:28.339 INFO:teuthology.orchestra.run.smithi076.stdout:4496 write 0x335aa5f thru 0x335fd82 (0x5324 bytes) 2024-09-14T22:54:28.340 INFO:teuthology.orchestra.run.smithi076.stdout:4497 write 0x831c144 thru 0x831c42b (0x2e8 bytes) 2024-09-14T22:54:28.340 INFO:teuthology.orchestra.run.smithi076.stdout:4498 write 0xd8f7d45 thru 0xd90247e (0xa73a bytes) 2024-09-14T22:54:28.340 INFO:teuthology.orchestra.run.smithi076.stdout:4499 read 0x34a33d3 thru 0x34a6b33 (0x3761 bytes) 2024-09-14T22:54:28.340 INFO:teuthology.orchestra.run.smithi076.stdout:4500 read 0x3e01805 thru 0x3e0a618 (0x8e14 bytes) 2024-09-14T22:54:28.340 INFO:teuthology.orchestra.run.smithi076.stdout:4501 read 0x2109a95 thru 0x210fa6d (0x5fd9 bytes) 2024-09-14T22:54:28.340 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.340 INFO:teuthology.orchestra.run.smithi076.stdout:4504 trunc from 0xd90247f to 0x2ea67da 2024-09-14T22:54:28.358 INFO:teuthology.orchestra.run.smithi076.stdout:4506 writesame 0x8d5d207 thru 0x8d5e61c data_size 0x3(0x1416 bytes) 2024-09-14T22:54:28.359 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.359 INFO:teuthology.orchestra.run.smithi076.stdout:4509 writesame 0x8555e1d thru 0x8557481 data_size 0x9(0x1665 bytes) 2024-09-14T22:54:28.362 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.362 INFO:teuthology.orchestra.run.smithi076.stdout:4511 punch from 0x6edc915 to 0x6ee567f, (0x8d6a bytes) 2024-09-14T22:54:28.373 INFO:teuthology.orchestra.run.smithi076.stdout:4512 write 0xcd115f0 thru 0xcd1786f (0x6280 bytes) 2024-09-14T22:54:28.377 INFO:teuthology.orchestra.run.smithi076.stdout:4514 write 0x49f4de thru 0x4a7506 (0x8029 bytes) 2024-09-14T22:54:28.378 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.383 INFO:teuthology.orchestra.run.smithi076.stdout:4516 trunc from 0xcd17870 to 0x9662b12 2024-09-14T22:54:28.403 INFO:teuthology.orchestra.run.smithi076.stdout:4518 writesame 0xac516c0 thru 0xac604c4 data_size 0x3(0xee05 bytes) 2024-09-14T22:54:28.404 INFO:teuthology.orchestra.run.smithi076.stdout:4519 write 0x4a3e31a thru 0x4a44b34 (0x681b bytes) 2024-09-14T22:54:28.406 INFO:teuthology.orchestra.run.smithi076.stdout:4521 read 0x21a6c10 thru 0x21b66bf (0xfab0 bytes) 2024-09-14T22:54:28.414 INFO:teuthology.orchestra.run.smithi076.stdout:4522 writesame 0xcb2bfbb thru 0xcb32342 data_size 0x46(0x6388 bytes) 2024-09-14T22:54:28.414 INFO:teuthology.orchestra.run.smithi076.stdout:4523 punch from 0x8fda045 to 0x8fe33a9, (0x9364 bytes) 2024-09-14T22:54:28.421 INFO:teuthology.orchestra.run.smithi076.stdout:4525 writesame 0xdfb04fe thru 0xdfb376f data_size 0xb(0x3272 bytes) 2024-09-14T22:54:28.421 INFO:teuthology.orchestra.run.smithi076.stdout:4526 write 0x9f775a9 thru 0x9f7812c (0xb84 bytes) 2024-09-14T22:54:28.425 INFO:teuthology.orchestra.run.smithi076.stdout:4527 read 0x26df94e thru 0x26ec947 (0xcffa bytes) 2024-09-14T22:54:28.425 INFO:teuthology.orchestra.run.smithi076.stdout:4528 writesame 0x618084e thru 0x618649b data_size 0x55(0x5c4e bytes) 2024-09-14T22:54:28.428 INFO:teuthology.orchestra.run.smithi076.stdout:4529 writesame 0x9d8ff83 thru 0x9d9aeec data_size 0x1(0xaf6a bytes) 2024-09-14T22:54:28.429 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:28.458 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:28.458 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:28.458 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:28.458 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:28.458 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:28.457+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:28.464 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.464 INFO:teuthology.orchestra.run.smithi076.stdout:4531 read 0x516022d thru 0x516cc25 (0xc9f9 bytes) 2024-09-14T22:54:28.465 INFO:teuthology.orchestra.run.smithi076.stdout:4534 write 0x41f57c5 thru 0x420414e (0xe98a bytes) 2024-09-14T22:54:28.467 INFO:teuthology.orchestra.run.smithi076.stdout:4535 read 0x4dac899 thru 0x4db3a2e (0x7196 bytes) 2024-09-14T22:54:28.468 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:28.469 INFO:teuthology.orchestra.run.smithi076.stdout:4536 write 0x25238c3 thru 0x252cf64 (0x96a2 bytes) 2024-09-14T22:54:28.470 INFO:teuthology.orchestra.run.smithi076.stdout:4537 read 0xd6a0085 thru 0xd6a234f (0x22cb bytes) 2024-09-14T22:54:28.471 INFO:teuthology.orchestra.run.smithi076.stdout:4538 read 0x1b10c98 thru 0x1b17a52 (0x6dbb bytes) 2024-09-14T22:54:28.471 INFO:teuthology.orchestra.run.smithi076.stdout:4540 punch from 0x3537ead to 0x353f682, (0x77d5 bytes) 2024-09-14T22:54:28.471 INFO:teuthology.orchestra.run.smithi076.stdout:4541 punch from 0x8b10712 to 0x8b1c5fc, (0xbeea bytes) 2024-09-14T22:54:28.473 INFO:teuthology.orchestra.run.smithi076.stdout:4542 trunc from 0xdfb3770 to 0xb2b0149 2024-09-14T22:54:28.484 INFO:teuthology.orchestra.run.smithi076.stdout:4544 writesame 0x490107 thru 0x49145f data_size 0xd(0x1359 bytes) 2024-09-14T22:54:28.485 INFO:teuthology.orchestra.run.smithi076.stdout:4545 writesame 0x8ff4d4 thru 0x909b29 data_size 0x6(0xa656 bytes) 2024-09-14T22:54:28.496 INFO:teuthology.orchestra.run.smithi076.stdout:4546 trunc from 0xb2b0149 to 0x7d86071 2024-09-14T22:54:28.515 INFO:teuthology.orchestra.run.smithi076.stdout:4550 writesame 0xae0162d thru 0xae0e11d data_size 0x1(0xcaf1 bytes) 2024-09-14T22:54:28.516 INFO:teuthology.orchestra.run.smithi076.stdout:4551 punch from 0x5a6f3ec to 0x5a72a7d, (0x3691 bytes) 2024-09-14T22:54:28.517 INFO:teuthology.orchestra.run.smithi076.stdout:4552 write 0x6e885ed thru 0x6e8910d (0xb21 bytes) 2024-09-14T22:54:28.519 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.519 INFO:teuthology.orchestra.run.smithi076.stdout:4554 read 0x2781c74 thru 0x278e7e4 (0xcb71 bytes) 2024-09-14T22:54:28.520 INFO:teuthology.orchestra.run.smithi076.stdout:4555 writesame 0x5dfcd71 thru 0x5e0613a data_size 0x1(0x93ca bytes) 2024-09-14T22:54:28.538 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:28.537+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:28.562 INFO:teuthology.orchestra.run.smithi076.stdout:4556 write 0x40f8687 thru 0x4102096 (0x9a10 bytes) 2024-09-14T22:54:28.564 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:28.564 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:54:28.565 INFO:teuthology.orchestra.run.smithi076.stdout:4557 punch from 0x23e3776 to 0x23e8e4a, (0x56d4 bytes) 2024-09-14T22:54:28.565 INFO:teuthology.orchestra.run.smithi076.stdout:4558 writesame 0x4353979 thru 0x435cf0c data_size 0x6(0x9594 bytes) 2024-09-14T22:54:28.570 INFO:teuthology.orchestra.run.smithi076.stdout:4559 trunc from 0xae0e11e to 0xa66a151 2024-09-14T22:54:28.578 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-14T22:54:28.169849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1452445657,"num_objects":2313,"num_object_clones":406,"num_object_copies":4626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2313,"num_whiteouts":1424,"num_read":214185,"num_read_kb":947827,"num_write":11728,"num_write_kb":506944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":34250838,"num_keys_recovered":18,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10849,"ondisk_log_size":10849,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":996372,"kb_used_data":677424,"kb_used_omap":12,"kb_used_meta":318707,"kb_avail":753978348,"statfs":{"total":773094113280,"available":772073828352,"internally_reserved":0,"allocated":693682176,"data_stored":804860628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":326356529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31770909,"num_objects":60,"num_object_clones":30,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":2,"num_read":-1817,"num_read_kb":-3225,"num_write":1136,"num_write_kb":-7803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-98,"num_bytes_recovered":-1352199,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027920"},"pg_stats":[{"pgid":"3.1f","version":"91'490","reported_seq":7543,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143791+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:54:24.143791+0000","last_peered":"2024-09-14T22:54:24.143791+0000","last_clean":"2024-09-14T22:54:24.143791+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:54:24.143791+0000","last_undegraded":"2024-09-14T22:54:24.143791+0000","last_fullsized":"2024-09-14T22:54:24.143791+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:21.625600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":45640341,"num_objects":79,"num_object_clones":17,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":47,"num_read":8754,"num_read_kb":28886,"num_write":579,"num_write_kb":6883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"unknown","last_fresh":"2024-09-14T22:54:24.145826+0000","last_change":"2024-09-14T22:54:24.819044+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:54:24.145826+0000","last_undegraded":"2024-09-14T22:54:24.145826+0000","last_fullsized":"2024-09-14T22:54:24.145826+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"90'276","reported_seq":7781,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.145100+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:54:24.145100+0000","last_peered":"2024-09-14T22:54:24.145100+0000","last_clean":"2024-09-14T22:54:24.145100+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:54:24.145100+0000","last_undegraded":"2024-09-14T22:54:24.145100+0000","last_fullsized":"2024-09-14T22:54:24.145100+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:06:32.916398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":55977828,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":47,"num_read":7944,"num_read_kb":34863,"num_write":266,"num_write_kb":32754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"91'410","reported_seq":2313,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144816+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:54:24.144816+0000","last_peered":"2024-09-14T22:54:24.144816+0000","last_clean":"2024-09-14T22:54:24.144816+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:54:24.144816+0000","last_undegraded":"2024-09-14T22:54:24.144816+0000","last_fullsized":"2024-09-14T22:54:24.144816+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:30:23.805939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":39407958,"num_objects":60,"num_object_clones":9,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":37,"num_read":3386,"num_read_kb":8895,"num_write":474,"num_write_kb":6537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"91'226","reported_seq":4268,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144105+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:54:24.144105+0000","last_peered":"2024-09-14T22:54:24.144105+0000","last_clean":"2024-09-14T22:54:24.144105+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:54:24.144105+0000","last_undegraded":"2024-09-14T22:54:24.144105+0000","last_fullsized":"2024-09-14T22:54:24.144105+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:39:51.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":76991017,"num_objects":68,"num_object_clones":9,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":41,"num_read":4028,"num_read_kb":50365,"num_write":231,"num_write_kb":17827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"91'335","reported_seq":11649,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148479+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:54:24.148479+0000","last_peered":"2024-09-14T22:54:24.148479+0000","last_clean":"2024-09-14T22:54:24.148479+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:54:24.148479+0000","last_undegraded":"2024-09-14T22:54:24.148479+0000","last_fullsized":"2024-09-14T22:54:24.148479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:06:27.925275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":59123683,"num_objects":95,"num_object_clones":24,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":53,"num_read":12125,"num_read_kb":48911,"num_write":277,"num_write_kb":28585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"90'293","reported_seq":15167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146232+0000","last_change":"2024-09-14T22:53:51.943155+0000","last_active":"2024-09-14T22:54:24.146232+0000","last_peered":"2024-09-14T22:54:24.146232+0000","last_clean":"2024-09-14T22:54:24.146232+0000","last_became_active":"2024-09-14T22:53:51.942663+0000","last_became_peered":"2024-09-14T22:53:51.942663+0000","last_unstale":"2024-09-14T22:54:24.146232+0000","last_undegraded":"2024-09-14T22:54:24.146232+0000","last_fullsized":"2024-09-14T22:54:24.146232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:42.672503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":75872371,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":58,"num_read":15236,"num_read_kb":91141,"num_write":260,"num_write_kb":37716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"unknown","last_fresh":"2024-09-14T22:54:24.145771+0000","last_change":"2024-09-14T22:54:24.819001+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:54:24.145771+0000","last_undegraded":"2024-09-14T22:54:24.145771+0000","last_fullsized":"2024-09-14T22:54:24.145771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"91'510","reported_seq":5967,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144867+0000","last_change":"2024-09-14T22:53:38.882316+0000","last_active":"2024-09-14T22:54:24.144867+0000","last_peered":"2024-09-14T22:54:24.144867+0000","last_clean":"2024-09-14T22:54:24.144867+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:54:24.144867+0000","last_undegraded":"2024-09-14T22:54:24.144867+0000","last_fullsized":"2024-09-14T22:54:24.144867+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:34.403347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":34010799,"num_objects":62,"num_object_clones":8,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":34,"num_read":7296,"num_read_kb":33358,"num_write":716,"num_write_kb":27062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"91'297","reported_seq":8684,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144852+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:54:24.144852+0000","last_peered":"2024-09-14T22:54:24.144852+0000","last_clean":"2024-09-14T22:54:24.144852+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:54:24.144852+0000","last_undegraded":"2024-09-14T22:54:24.144852+0000","last_fullsized":"2024-09-14T22:54:24.144852+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:50:15.160158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":61201000,"num_objects":77,"num_object_clones":17,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":48,"num_read":9460,"num_read_kb":43214,"num_write":257,"num_write_kb":15493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'211","reported_seq":8536,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146549+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:54:24.146549+0000","last_peered":"2024-09-14T22:54:24.146549+0000","last_clean":"2024-09-14T22:54:24.146549+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:54:24.146549+0000","last_undegraded":"2024-09-14T22:54:24.146549+0000","last_fullsized":"2024-09-14T22:54:24.146549+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:50.097468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":34741299,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":42,"num_read":8949,"num_read_kb":41372,"num_write":163,"num_write_kb":13237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"91'400","reported_seq":10339,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148343+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:54:24.148343+0000","last_peered":"2024-09-14T22:54:24.148343+0000","last_clean":"2024-09-14T22:54:24.148343+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:54:24.148343+0000","last_undegraded":"2024-09-14T22:54:24.148343+0000","last_fullsized":"2024-09-14T22:54:24.148343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:42.548136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":94275451,"num_objects":91,"num_object_clones":18,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":49,"num_read":10944,"num_read_kb":65418,"num_write":468,"num_write_kb":19420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143228+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:54:24.143228+0000","last_peered":"2024-09-14T22:54:24.143228+0000","last_clean":"2024-09-14T22:54:24.143228+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:54:24.143228+0000","last_undegraded":"2024-09-14T22:54:24.143228+0000","last_fullsized":"2024-09-14T22:54:24.143228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"91'507","reported_seq":7116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144555+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:54:24.144555+0000","last_peered":"2024-09-14T22:54:24.144555+0000","last_clean":"2024-09-14T22:54:24.144555+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:54:24.144555+0000","last_undegraded":"2024-09-14T22:54:24.144555+0000","last_fullsized":"2024-09-14T22:54:24.144555+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:17:07.382895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":73946266,"num_objects":69,"num_object_clones":11,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":39,"num_read":8733,"num_read_kb":39019,"num_write":604,"num_write_kb":23567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144961+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:54:24.144961+0000","last_peered":"2024-09-14T22:54:24.144961+0000","last_clean":"2024-09-14T22:54:24.144961+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:54:24.144961+0000","last_undegraded":"2024-09-14T22:54:24.144961+0000","last_fullsized":"2024-09-14T22:54:24.144961+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"91'194","reported_seq":1264,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146746+0000","last_change":"2024-09-14T22:54:07.044085+0000","last_active":"2024-09-14T22:54:24.146746+0000","last_peered":"2024-09-14T22:54:24.146746+0000","last_clean":"2024-09-14T22:54:24.146746+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:54:24.146746+0000","last_undegraded":"2024-09-14T22:54:24.146746+0000","last_fullsized":"2024-09-14T22:54:24.146746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:45:02.350894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":37076310,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":41,"num_read":885,"num_read_kb":11514,"num_write":183,"num_write_kb":14876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146646+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:54:24.146646+0000","last_peered":"2024-09-14T22:54:24.146646+0000","last_clean":"2024-09-14T22:54:24.146646+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:54:24.146646+0000","last_undegraded":"2024-09-14T22:54:24.146646+0000","last_fullsized":"2024-09-14T22:54:24.146646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"91'528","reported_seq":12864,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.147130+0000","last_change":"2024-09-14T22:53:51.942879+0000","last_active":"2024-09-14T22:54:24.147130+0000","last_peered":"2024-09-14T22:54:24.147130+0000","last_clean":"2024-09-14T22:54:24.147130+0000","last_became_active":"2024-09-14T22:53:51.942267+0000","last_became_peered":"2024-09-14T22:53:51.942267+0000","last_unstale":"2024-09-14T22:54:24.147130+0000","last_undegraded":"2024-09-14T22:54:24.147130+0000","last_fullsized":"2024-09-14T22:54:24.147130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:49.720399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":33598550,"num_objects":89,"num_object_clones":15,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":54,"num_read":14203,"num_read_kb":38154,"num_write":674,"num_write_kb":19221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143569+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:54:24.143569+0000","last_peered":"2024-09-14T22:54:24.143569+0000","last_clean":"2024-09-14T22:54:24.143569+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:54:24.143569+0000","last_undegraded":"2024-09-14T22:54:24.143569+0000","last_fullsized":"2024-09-14T22:54:24.143569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"91'303","reported_seq":14311,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.206854+0000","last_change":"2024-09-14T22:54:24.145705+0000","last_active":"2024-09-14T22:54:24.145705+0000","last_peered":"2024-09-14T22:54:23.206854+0000","last_clean":"2024-09-14T22:54:23.206854+0000","last_became_active":"2024-09-14T22:53:35.868554+0000","last_became_peered":"2024-09-14T22:53:35.868554+0000","last_unstale":"2024-09-14T22:54:23.206854+0000","last_undegraded":"2024-09-14T22:54:23.206854+0000","last_fullsized":"2024-09-14T22:54:23.206854+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":74854667,"num_objects":86,"num_object_clones":15,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":50,"num_read":13533,"num_read_kb":47864,"num_write":276,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":3794613,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"91'353","reported_seq":5121,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148539+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:54:24.148539+0000","last_peered":"2024-09-14T22:54:24.148539+0000","last_clean":"2024-09-14T22:54:24.148539+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:54:24.148539+0000","last_undegraded":"2024-09-14T22:54:24.148539+0000","last_fullsized":"2024-09-14T22:54:24.148539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:22:37.321498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":53351379,"num_objects":89,"num_object_clones":17,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":57,"num_read":6535,"num_read_kb":53355,"num_write":314,"num_write_kb":30002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144716+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:54:24.144716+0000","last_peered":"2024-09-14T22:54:24.144716+0000","last_clean":"2024-09-14T22:54:24.144716+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:54:24.144716+0000","last_undegraded":"2024-09-14T22:54:24.144716+0000","last_fullsized":"2024-09-14T22:54:24.144716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"91'223","reported_seq":1004,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143161+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:54:24.143161+0000","last_peered":"2024-09-14T22:54:24.143161+0000","last_clean":"2024-09-14T22:54:24.143161+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:54:24.143161+0000","last_undegraded":"2024-09-14T22:54:24.143161+0000","last_fullsized":"2024-09-14T22:54:24.143161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:34.116593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":12575758,"num_objects":58,"num_object_clones":8,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":981,"num_read_kb":7915,"num_write":230,"num_write_kb":11010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.187759+0000","last_change":"2024-09-14T22:54:24.146834+0000","last_active":"2024-09-14T22:54:24.146834+0000","last_peered":"2024-09-14T22:54:23.187759+0000","last_clean":"2024-09-14T22:54:23.187759+0000","last_became_active":"2024-09-14T22:53:32.745120+0000","last_became_peered":"2024-09-14T22:53:32.745120+0000","last_unstale":"2024-09-14T22:54:23.187759+0000","last_undegraded":"2024-09-14T22:54:23.187759+0000","last_fullsized":"2024-09-14T22:54:23.187759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"91'458","reported_seq":8405,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144171+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:54:24.144171+0000","last_peered":"2024-09-14T22:54:24.144171+0000","last_clean":"2024-09-14T22:54:24.144171+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:54:24.144171+0000","last_undegraded":"2024-09-14T22:54:24.144171+0000","last_fullsized":"2024-09-14T22:54:24.144171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:51:32.597165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":26518013,"num_objects":73,"num_object_clones":11,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":43,"num_read":9613,"num_read_kb":20323,"num_write":595,"num_write_kb":10180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146386+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:54:24.146386+0000","last_peered":"2024-09-14T22:54:24.146386+0000","last_clean":"2024-09-14T22:54:24.146386+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:54:24.146386+0000","last_undegraded":"2024-09-14T22:54:24.146386+0000","last_fullsized":"2024-09-14T22:54:24.146386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'259","reported_seq":5575,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148312+0000","last_change":"2024-09-14T22:54:18.745625+0000","last_active":"2024-09-14T22:54:24.148312+0000","last_peered":"2024-09-14T22:54:24.148312+0000","last_clean":"2024-09-14T22:54:24.148312+0000","last_became_active":"2024-09-14T22:54:18.744069+0000","last_became_peered":"2024-09-14T22:54:18.744069+0000","last_unstale":"2024-09-14T22:54:24.148312+0000","last_undegraded":"2024-09-14T22:54:24.148312+0000","last_fullsized":"2024-09-14T22:54:24.148312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:27.211239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":58509638,"num_objects":92,"num_object_clones":16,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":59,"num_read":4963,"num_read_kb":28758,"num_write":210,"num_write_kb":28454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":160,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144716+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:54:24.144716+0000","last_peered":"2024-09-14T22:54:24.144716+0000","last_clean":"2024-09-14T22:54:24.144716+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:54:24.144716+0000","last_undegraded":"2024-09-14T22:54:24.144716+0000","last_fullsized":"2024-09-14T22:54:24.144716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":310,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146453+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:54:24.146453+0000","last_peered":"2024-09-14T22:54:24.146453+0000","last_clean":"2024-09-14T22:54:24.146453+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:54:24.146453+0000","last_undegraded":"2024-09-14T22:54:24.146453+0000","last_fullsized":"2024-09-14T22:54:24.146453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"91'431","reported_seq":8221,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146404+0000","last_change":"2024-09-14T22:53:48.924907+0000","last_active":"2024-09-14T22:54:24.146404+0000","last_peered":"2024-09-14T22:54:24.146404+0000","last_clean":"2024-09-14T22:54:24.146404+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:54:24.146404+0000","last_undegraded":"2024-09-14T22:54:24.146404+0000","last_fullsized":"2024-09-14T22:54:24.146404+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:03:29.838215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":49431084,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":57,"num_read":10404,"num_read_kb":32163,"num_write":390,"num_write_kb":12535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"91'281","reported_seq":1287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148342+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:54:24.148342+0000","last_peered":"2024-09-14T22:54:24.148342+0000","last_clean":"2024-09-14T22:54:24.148342+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:54:24.148342+0000","last_undegraded":"2024-09-14T22:54:24.148342+0000","last_fullsized":"2024-09-14T22:54:24.148342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:45:15.048406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":22447246,"num_objects":84,"num_object_clones":15,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":57,"num_read":1795,"num_read_kb":6529,"num_write":246,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'345","reported_seq":1365,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143494+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:54:24.143494+0000","last_peered":"2024-09-14T22:54:24.143494+0000","last_clean":"2024-09-14T22:54:24.143494+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:54:24.143494+0000","last_undegraded":"2024-09-14T22:54:24.143494+0000","last_fullsized":"2024-09-14T22:54:24.143494+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:44.462443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":31221787,"num_objects":62,"num_object_clones":9,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":38,"num_read":2278,"num_read_kb":7206,"num_write":374,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"91'302","reported_seq":1934,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146253+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:54:24.146253+0000","last_peered":"2024-09-14T22:54:24.146253+0000","last_clean":"2024-09-14T22:54:24.146253+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:54:24.146253+0000","last_undegraded":"2024-09-14T22:54:24.146253+0000","last_fullsized":"2024-09-14T22:54:24.146253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:53:07.482998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":30215193,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":53,"num_read":2205,"num_read_kb":16804,"num_write":284,"num_write_kb":8479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"91'430","reported_seq":4684,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.325303+0000","last_change":"2024-09-14T22:54:24.145350+0000","last_active":"2024-09-14T22:54:24.145350+0000","last_peered":"2024-09-14T22:54:23.325303+0000","last_clean":"2024-09-14T22:54:23.325303+0000","last_became_active":"2024-09-14T22:53:35.868711+0000","last_became_peered":"2024-09-14T22:53:35.868711+0000","last_unstale":"2024-09-14T22:54:23.325303+0000","last_undegraded":"2024-09-14T22:54:23.325303+0000","last_fullsized":"2024-09-14T22:54:23.325303+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":32113789,"num_objects":79,"num_object_clones":9,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":49,"num_read":5574,"num_read_kb":20143,"num_write":458,"num_write_kb":16740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1667203,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"91'252","reported_seq":6115,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146644+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:54:24.146644+0000","last_peered":"2024-09-14T22:54:24.146644+0000","last_clean":"2024-09-14T22:54:24.146644+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:54:24.146644+0000","last_undegraded":"2024-09-14T22:54:24.146644+0000","last_fullsized":"2024-09-14T22:54:24.146644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:51.281278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":26665337,"num_objects":75,"num_object_clones":9,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":51,"num_read":6022,"num_read_kb":32214,"num_write":264,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"91'341","reported_seq":6296,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146744+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:54:24.146744+0000","last_peered":"2024-09-14T22:54:24.146744+0000","last_clean":"2024-09-14T22:54:24.146744+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:54:24.146744+0000","last_undegraded":"2024-09-14T22:54:24.146744+0000","last_fullsized":"2024-09-14T22:54:24.146744+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:40:33.245308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":56560750,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":47,"num_read":7166,"num_read_kb":39420,"num_write":329,"num_write_kb":20699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"91'228","reported_seq":5855,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146760+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:54:24.146760+0000","last_peered":"2024-09-14T22:54:24.146760+0000","last_clean":"2024-09-14T22:54:24.146760+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:54:24.146760+0000","last_undegraded":"2024-09-14T22:54:24.146760+0000","last_fullsized":"2024-09-14T22:54:24.146760+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:39.489218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":28566586,"num_objects":76,"num_object_clones":14,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":46,"num_read":5622,"num_read_kb":20719,"num_write":204,"num_write_kb":11550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"91'336","reported_seq":8846,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143312+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:54:24.143312+0000","last_peered":"2024-09-14T22:54:24.143312+0000","last_clean":"2024-09-14T22:54:24.143312+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:54:24.143312+0000","last_undegraded":"2024-09-14T22:54:24.143312+0000","last_fullsized":"2024-09-14T22:54:24.143312+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:47:33.886663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":68945901,"num_objects":77,"num_object_clones":11,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":46,"num_read":10314,"num_read_kb":28498,"num_write":320,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"90'439","reported_seq":4893,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146481+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:54:24.146481+0000","last_peered":"2024-09-14T22:54:24.146481+0000","last_clean":"2024-09-14T22:54:24.146481+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:54:24.146481+0000","last_undegraded":"2024-09-14T22:54:24.146481+0000","last_fullsized":"2024-09-14T22:54:24.146481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:31.691503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":54375844,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":41,"num_read":5815,"num_read_kb":32416,"num_write":575,"num_write_kb":22148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"91'309","reported_seq":1086,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146842+0000","last_change":"2024-09-14T22:53:38.883182+0000","last_active":"2024-09-14T22:54:24.146842+0000","last_peered":"2024-09-14T22:54:24.146842+0000","last_clean":"2024-09-14T22:54:24.146842+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:54:24.146842+0000","last_undegraded":"2024-09-14T22:54:24.146842+0000","last_fullsized":"2024-09-14T22:54:24.146842+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.546351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":47449396,"num_objects":71,"num_object_clones":11,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":50,"num_read":1181,"num_read_kb":9497,"num_write":358,"num_write_kb":9079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"92'762","reported_seq":5791,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146607+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:54:24.146607+0000","last_peered":"2024-09-14T22:54:24.146607+0000","last_clean":"2024-09-14T22:54:24.146607+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:54:24.146607+0000","last_undegraded":"2024-09-14T22:54:24.146607+0000","last_fullsized":"2024-09-14T22:54:24.146607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:11:57.358507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":56190029,"num_objects":84,"num_object_clones":13,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":56,"num_read":8125,"num_read_kb":8793,"num_write":988,"num_write_kb":3875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1451855270,"num_objects":2309,"num_object_clones":406,"num_object_copies":4618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2309,"num_whiteouts":1424,"num_read":214069,"num_read_kb":947727,"num_write":11567,"num_write_kb":504526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":34250838,"num_keys_recovered":18,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":683827200,"data_stored":799669840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10729,"ondisk_log_size":10729,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134648,"kb_used_data":90980,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94237192,"statfs":{"total":96636764160,"available":96498884608,"internally_reserved":0,"allocated":93163520,"data_stored":107519932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89832,"kb_used_data":54476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94282008,"statfs":{"total":96636764160,"available":96544776192,"internally_reserved":0,"allocated":55783424,"data_stored":63670191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121124,"kb_used_data":81568,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94250716,"statfs":{"total":96636764160,"available":96512733184,"internally_reserved":0,"allocated":83525632,"data_stored":92188032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206580,"kb_used_data":157340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94165260,"statfs":{"total":96636764160,"available":96425226240,"internally_reserved":0,"allocated":161116160,"data_stored":182895675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84732,"kb_used_data":49380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94287108,"statfs":{"total":96636764160,"available":96549998592,"internally_reserved":0,"allocated":50565120,"data_stored":55192059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79400,"kb_used_data":44044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94292440,"statfs":{"total":96636764160,"available":96555458560,"internally_reserved":0,"allocated":45101056,"data_stored":56158569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123052,"kb_used_data":87704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94248788,"statfs":{"total":96636764160,"available":96510758912,"internally_reserved":0,"allocated":89808896,"data_stored":116327143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157004,"kb_used_data":111932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94214836,"statfs":{"total":96636764160,"available":96475992064,"internally_reserved":0,"allocated":114618368,"data_stored":130909027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":112951296,"data_stored":129822208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88719360,"data_stored":115824272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":44015616,"data_stored":55655717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49479680,"data_stored":54689207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":160030720,"data_stored":182392823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":82436096,"data_stored":91685161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":54697984,"data_stored":63167339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":91496448,"data_stored":106433113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:54:28.580 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:54:28.581 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.581 INFO:teuthology.orchestra.run.smithi076.stdout:4561 punch from 0x34b0f4c to 0x34b7d3a, (0x6dee bytes) 2024-09-14T22:54:28.581 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.582 INFO:teuthology.orchestra.run.smithi076.stdout:4566 read 0xefe749 thru 0xf001f6 (0x1aae bytes) 2024-09-14T22:54:28.585 INFO:teuthology.orchestra.run.smithi076.stdout:4568 write 0xa99d879 thru 0xa9ad2e7 (0xfa6f bytes) 2024-09-14T22:54:28.587 INFO:teuthology.orchestra.run.smithi076.stdout:4569 read 0x6d2027f thru 0x6d24d27 (0x4aa9 bytes) 2024-09-14T22:54:28.589 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:28.590 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:28.597 INFO:teuthology.orchestra.run.smithi076.stdout:4570 writesame 0xd42167f thru 0xd42c914 data_size 0x1(0xb296 bytes) 2024-09-14T22:54:28.597 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.597 INFO:teuthology.orchestra.run.smithi076.stdout:4572 write 0xa0ccb7a thru 0xa0d3e26 (0x72ad bytes) 2024-09-14T22:54:28.598 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.598 INFO:teuthology.orchestra.run.smithi076.stdout:4574 write 0x432fd6d thru 0x433199f (0x1c33 bytes) 2024-09-14T22:54:28.599 INFO:teuthology.orchestra.run.smithi076.stdout:4575 write 0x22daf2b thru 0x22e2ba6 (0x7c7c bytes) 2024-09-14T22:54:28.601 INFO:teuthology.orchestra.run.smithi076.stdout:4576 writesame 0x378664f thru 0x3794ed4 data_size 0x9(0xe886 bytes) 2024-09-14T22:54:28.605 INFO:teuthology.orchestra.run.smithi076.stdout:4578 write 0x7f1c9d6 thru 0x7f2442d (0x7a58 bytes) 2024-09-14T22:54:28.607 INFO:teuthology.orchestra.run.smithi076.stdout:4579 write 0x675be04 thru 0x6767964 (0xbb61 bytes) 2024-09-14T22:54:28.608 INFO:teuthology.orchestra.run.smithi076.stdout:4580 read 0xbf77e17 thru 0xbf7ce5d (0x5047 bytes) 2024-09-14T22:54:28.608 INFO:teuthology.orchestra.run.smithi076.stdout:4581 flatten 2024-09-14T22:54:28.614 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:28.614 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:28.616 INFO:teuthology.orchestra.run.smithi076.stdout:4582 writesame 0x382dc98 thru 0x38363e3 data_size 0xe(0x874c bytes) 2024-09-14T22:54:28.619 INFO:teuthology.orchestra.run.smithi076.stdout:4583 write 0x4eb775e thru 0x4ec65d1 (0xee74 bytes) 2024-09-14T22:54:28.621 INFO:teuthology.orchestra.run.smithi076.stdout:4584 read 0x1850e84 thru 0x185da26 (0xcba3 bytes) 2024-09-14T22:54:28.625 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:28.625 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:28.627 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:28.627 INFO:teuthology.orchestra.run.smithi076.stdout:4585 trunc from 0xd42c915 to 0x9501d1b 2024-09-14T22:54:28.634 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.634 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.634 INFO:teuthology.orchestra.run.smithi076.stdout:4588 read 0x13204db thru 0x132e8f1 (0xe417 bytes) 2024-09-14T22:54:28.635 INFO:teuthology.orchestra.run.smithi076.stdout:4589 punch from 0x1087efd to 0x1088196, (0x299 bytes) 2024-09-14T22:54:28.635 INFO:teuthology.orchestra.run.smithi076.stdout:4590 write 0x5c2a56d thru 0x5c3a3c3 (0xfe57 bytes) 2024-09-14T22:54:28.637 INFO:teuthology.orchestra.run.smithi076.stdout:4592 punch from 0x6e24058 to 0x6e319fe, (0xd9a6 bytes) 2024-09-14T22:54:28.637 INFO:teuthology.orchestra.run.smithi076.stdout:4593 punch from 0x3c6f81c to 0x3c7e215, (0xe9f9 bytes) 2024-09-14T22:54:28.637 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.637 INFO:teuthology.orchestra.run.smithi076.stdout:4596 write 0x6882ac thru 0x697855 (0xf5aa bytes) 2024-09-14T22:54:28.638 INFO:teuthology.orchestra.run.smithi076.stdout:4598 write 0x1f21d3c thru 0x1f22a64 (0xd29 bytes) 2024-09-14T22:54:28.639 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:28.637+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:28.639 INFO:teuthology.orchestra.run.smithi076.stdout:4599 read 0x600792e thru 0x600bfd2 (0x46a5 bytes) 2024-09-14T22:54:28.640 INFO:teuthology.orchestra.run.smithi076.stdout:4600 read 0xd016ff thru 0xd0982f (0x8131 bytes) 2024-09-14T22:54:28.642 INFO:teuthology.orchestra.run.smithi076.stdout:4602 trunc from 0x9501d1b to 0x81abb23 2024-09-14T22:54:28.644 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:28.644 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:28.650 INFO:teuthology.orchestra.run.smithi076.stdout:4603 write 0x4b709a1 thru 0x4b76d02 (0x6362 bytes) 2024-09-14T22:54:28.651 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.652 INFO:teuthology.orchestra.run.smithi076.stdout:4606 writesame 0x5c06988 thru 0x5c0d628 data_size 0x1(0x6ca1 bytes) 2024-09-14T22:54:28.669 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:28.669 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:28.675 INFO:teuthology.orchestra.run.smithi076.stdout:4607 trunc from 0x81abb23 to 0x825233c 2024-09-14T22:54:28.681 INFO:teuthology.orchestra.run.smithi076.stdout:4612 punch from 0x7a954b8 to 0x7aa17e9, (0xc331 bytes) 2024-09-14T22:54:28.682 INFO:teuthology.orchestra.run.smithi076.stdout:4613 write 0x49617f9 thru 0x4965039 (0x3841 bytes) 2024-09-14T22:54:28.683 INFO:teuthology.orchestra.run.smithi076.stdout:4616 read 0x6254dc1 thru 0x6256b6c (0x1dac bytes) 2024-09-14T22:54:28.688 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:28.688 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:28.696 INFO:teuthology.orchestra.run.smithi076.stdout:4617 writesame 0xc908048 thru 0xc90a39d data_size 0x1(0x2356 bytes) 2024-09-14T22:54:28.696 INFO:teuthology.orchestra.run.smithi076.stdout:4619 write 0x528d6ae thru 0x5290143 (0x2a96 bytes) 2024-09-14T22:54:28.697 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.697 INFO:teuthology.orchestra.run.smithi076.stdout:4622 read 0x887cf0e thru 0x888a14a (0xd23d bytes) 2024-09-14T22:54:28.697 INFO:teuthology.orchestra.run.smithi076.stdout:4623 read 0x4b351ce thru 0x4b3d713 (0x8546 bytes) 2024-09-14T22:54:28.698 INFO:teuthology.orchestra.run.smithi076.stdout:4624 writesame 0x35684d9 thru 0x3569d70 data_size 0x4(0x1898 bytes) 2024-09-14T22:54:28.700 INFO:teuthology.orchestra.run.smithi076.stdout:4625 read 0xb2e6172 thru 0xb2f31c6 (0xd055 bytes) 2024-09-14T22:54:28.708 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:28.708 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:28.710 INFO:teuthology.orchestra.run.smithi076.stdout:4627 writesame 0xe83c95c thru 0xe83f43a data_size 0x5(0x2adf bytes) 2024-09-14T22:54:28.710 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.710 INFO:teuthology.orchestra.run.smithi076.stdout:4630 punch from 0x7789392 to 0x779124a, (0x7eb8 bytes) 2024-09-14T22:54:28.710 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.710 INFO:teuthology.orchestra.run.smithi076.stdout:4632 read 0xd73d9c7 thru 0xd7443be (0x69f8 bytes) 2024-09-14T22:54:28.727 INFO:teuthology.orchestra.run.smithi076.stdout:4633 trunc from 0xe83f43b to 0x1d52b53 2024-09-14T22:54:28.737 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:28.738 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:28.740 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:28.738+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:28.744 INFO:teuthology.orchestra.run.smithi076.stdout:4634 punch from 0x13f349a to 0x13fa749, (0x72af bytes) 2024-09-14T22:54:28.744 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.750 INFO:teuthology.orchestra.run.smithi076.stdout:4636 trunc from 0x1d52b53 to 0x66605df 2024-09-14T22:54:28.756 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.762 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:28.762 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:28.765 INFO:teuthology.orchestra.run.smithi076.stdout:4640 writesame 0x8daa0cf thru 0x8daec89 data_size 0x1(0x4bbb bytes) 2024-09-14T22:54:28.765 INFO:teuthology.orchestra.run.smithi076.stdout:4641 read 0x76e6abd thru 0x76ece04 (0x6348 bytes) 2024-09-14T22:54:28.765 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.766 INFO:teuthology.orchestra.run.smithi076.stdout:4643 write 0x57648ff thru 0x57736c6 (0xedc8 bytes) 2024-09-14T22:54:28.767 INFO:teuthology.orchestra.run.smithi076.stdout:4644 read 0x70793b1 thru 0x707f02c (0x5c7c bytes) 2024-09-14T22:54:28.767 INFO:teuthology.orchestra.run.smithi076.stdout:4645 read 0x5cde894 thru 0x5ce004d (0x17ba bytes) 2024-09-14T22:54:28.768 INFO:teuthology.orchestra.run.smithi076.stdout:4646 write 0x68ea899 thru 0x68f7d68 (0xd4d0 bytes) 2024-09-14T22:54:28.770 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:28.770 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:28.782 INFO:teuthology.orchestra.run.smithi076.stdout:4647 write 0xd6d8760 thru 0xd6e6681 (0xdf22 bytes) 2024-09-14T22:54:28.785 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.785 INFO:teuthology.orchestra.run.smithi076.stdout:4650 read 0x17330c8 thru 0x173d21d (0xa156 bytes) 2024-09-14T22:54:28.786 INFO:teuthology.orchestra.run.smithi076.stdout:4653 read 0xc06c479 thru 0xc0764da (0xa062 bytes) 2024-09-14T22:54:28.786 INFO:teuthology.orchestra.run.smithi076.stdout:4654 punch from 0x2c3f2aa to 0x2c4ef17, (0xfc6d bytes) 2024-09-14T22:54:28.786 INFO:teuthology.orchestra.run.smithi076.stdout:4656 punch from 0xaa279b5 to 0xaa314c8, (0x9b13 bytes) 2024-09-14T22:54:28.786 INFO:teuthology.orchestra.run.smithi076.stdout:4657 read 0xf7e07c thru 0xf88286 (0xa20b bytes) 2024-09-14T22:54:28.787 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:28.787 INFO:teuthology.orchestra.run.smithi076.stdout:4658 write 0x810e935 thru 0x811b264 (0xc930 bytes) 2024-09-14T22:54:28.790 INFO:teuthology.orchestra.run.smithi076.stdout:4660 punch from 0xa5c76dc to 0xa5cc4c6, (0x4dea bytes) 2024-09-14T22:54:28.790 INFO:teuthology.orchestra.run.smithi076.stdout:4661 read 0x22fb278 thru 0x2303157 (0x7ee0 bytes) 2024-09-14T22:54:28.790 INFO:teuthology.orchestra.run.smithi076.stdout:4663 write 0xb64ab4b thru 0xb652079 (0x752f bytes) 2024-09-14T22:54:28.793 INFO:teuthology.orchestra.run.smithi076.stdout:4664 read 0x846bc0c thru 0x847801b (0xc410 bytes) 2024-09-14T22:54:28.793 INFO:teuthology.orchestra.run.smithi076.stdout:4665 write 0xca80a80 thru 0xca87d61 (0x72e2 bytes) 2024-09-14T22:54:28.794 INFO:teuthology.orchestra.run.smithi076.stdout:4666 read 0x705d30b thru 0x70677a3 (0xa499 bytes) 2024-09-14T22:54:28.804 INFO:teuthology.orchestra.run.smithi076.stdout:4669 trunc from 0xd6e6682 to 0x56edfcd 2024-09-14T22:54:28.817 INFO:teuthology.orchestra.run.smithi076.stdout:4672 read 0x360474f thru 0x3609b52 (0x5404 bytes) 2024-09-14T22:54:28.817 INFO:teuthology.orchestra.run.smithi076.stdout:4673 write 0x1032beb thru 0x103a637 (0x7a4d bytes) 2024-09-14T22:54:28.819 INFO:teuthology.orchestra.run.smithi076.stdout:4675 punch from 0x1f8049c to 0x1f8628e, (0x5df2 bytes) 2024-09-14T22:54:28.819 INFO:teuthology.orchestra.run.smithi076.stdout:4676 read 0x10c38da thru 0x10c9550 (0x5c77 bytes) 2024-09-14T22:54:28.825 INFO:teuthology.orchestra.run.smithi076.stdout:4677 write 0x5ed1ea0 thru 0x5ed2e98 (0xff9 bytes) 2024-09-14T22:54:28.827 INFO:teuthology.orchestra.run.smithi076.stdout:4678 read 0x278ef3a thru 0x279644e (0x7515 bytes) 2024-09-14T22:54:28.827 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.829 INFO:teuthology.orchestra.run.smithi076.stdout:4683 trunc from 0x5ed2e99 to 0x78229ca 2024-09-14T22:54:28.839 INFO:teuthology.orchestra.run.smithi076.stdout:4685 trunc from 0x78229ca to 0x4983f88 2024-09-14T22:54:28.841 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:28.839+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:28.849 INFO:teuthology.orchestra.run.smithi076.stdout:4687 punch from 0x3ce504e to 0x3cea620, (0x55d2 bytes) 2024-09-14T22:54:28.853 INFO:teuthology.orchestra.run.smithi076.stdout:4688 trunc from 0x4983f88 to 0x10f27f8 2024-09-14T22:54:28.863 INFO:teuthology.orchestra.run.smithi076.stdout:4689 read 0x10b63c4 thru 0x10b63c4 (0x1 bytes) 2024-09-14T22:54:28.863 INFO:teuthology.orchestra.run.smithi076.stdout:4690 read 0xcb8ad9 thru 0xcbe00d (0x5535 bytes) 2024-09-14T22:54:28.875 INFO:teuthology.orchestra.run.smithi076.stdout:4691 write 0x54c658e thru 0x54c996c (0x33df bytes) 2024-09-14T22:54:28.876 INFO:teuthology.orchestra.run.smithi076.stdout:4692 punch from 0x40f4f22 to 0x4104556, (0xf634 bytes) 2024-09-14T22:54:28.876 INFO:teuthology.orchestra.run.smithi076.stdout:4693 punch from 0x1105e2 to 0x11179f, (0x11bd bytes) 2024-09-14T22:54:28.876 INFO:teuthology.orchestra.run.smithi076.stdout:4695 read 0x4cb3703 thru 0x4cb3a06 (0x304 bytes) 2024-09-14T22:54:28.877 INFO:teuthology.orchestra.run.smithi076.stdout:4696 trunc from 0x54c996d to 0x463155e 2024-09-14T22:54:28.885 INFO:teuthology.orchestra.run.smithi076.stdout:4697 write 0x13e550 thru 0x14d57d (0xf02e bytes) 2024-09-14T22:54:28.896 INFO:teuthology.orchestra.run.smithi076.stdout:4698 writesame 0x6f390bc thru 0x6f44980 data_size 0x1(0xb8c5 bytes) 2024-09-14T22:54:28.899 INFO:teuthology.orchestra.run.smithi076.stdout:4699 trunc from 0x6f44981 to 0x988f8f7 2024-09-14T22:54:28.905 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.916 INFO:teuthology.orchestra.run.smithi076.stdout:4701 write 0xe1fd240 thru 0xe20add1 (0xdb92 bytes) 2024-09-14T22:54:28.918 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:28.918 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:28.920 INFO:teuthology.orchestra.run.smithi076.stdout:4702 write 0x1662fe3 thru 0x1669953 (0x6971 bytes) 2024-09-14T22:54:28.922 INFO:teuthology.orchestra.run.smithi076.stdout:4703 writesame 0xb148db9 thru 0xb14b7fd data_size 0x1(0x2a45 bytes) 2024-09-14T22:54:28.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:28.930 INFO:teuthology.orchestra.run.smithi076.stdout:4704 read 0x4ac87c6 thru 0x4ad3f46 (0xb781 bytes) 2024-09-14T22:54:28.936 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:28.936 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:54:28.941 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:28.940+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:28.946 INFO:teuthology.orchestra.run.smithi076.stdout:4708 trunc from 0xe20add2 to 0x1ada133 2024-09-14T22:54:28.954 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-14T22:54:28.798505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1452445657,"num_objects":2313,"num_object_clones":406,"num_object_copies":4626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2313,"num_whiteouts":1424,"num_read":214185,"num_read_kb":947827,"num_write":11728,"num_write_kb":506944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":34250838,"num_keys_recovered":18,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10849,"ondisk_log_size":10849,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":996372,"kb_used_data":677424,"kb_used_omap":12,"kb_used_meta":318707,"kb_avail":753978348,"statfs":{"total":773094113280,"available":772073828352,"internally_reserved":0,"allocated":693682176,"data_stored":804860628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":326356529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31770909,"num_objects":60,"num_object_clones":30,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":2,"num_read":-1817,"num_read_kb":-3225,"num_write":1136,"num_write_kb":-7803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-98,"num_bytes_recovered":-1352199,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000625"},"pg_stats":[{"pgid":"3.1f","version":"91'490","reported_seq":7543,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143791+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:54:24.143791+0000","last_peered":"2024-09-14T22:54:24.143791+0000","last_clean":"2024-09-14T22:54:24.143791+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:54:24.143791+0000","last_undegraded":"2024-09-14T22:54:24.143791+0000","last_fullsized":"2024-09-14T22:54:24.143791+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:21.625600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":45640341,"num_objects":79,"num_object_clones":17,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":47,"num_read":8754,"num_read_kb":28886,"num_write":579,"num_write_kb":6883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"unknown","last_fresh":"2024-09-14T22:54:24.145826+0000","last_change":"2024-09-14T22:54:24.819044+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:54:24.145826+0000","last_undegraded":"2024-09-14T22:54:24.145826+0000","last_fullsized":"2024-09-14T22:54:24.145826+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"90'276","reported_seq":7781,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.145100+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:54:24.145100+0000","last_peered":"2024-09-14T22:54:24.145100+0000","last_clean":"2024-09-14T22:54:24.145100+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:54:24.145100+0000","last_undegraded":"2024-09-14T22:54:24.145100+0000","last_fullsized":"2024-09-14T22:54:24.145100+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:06:32.916398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":55977828,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":47,"num_read":7944,"num_read_kb":34863,"num_write":266,"num_write_kb":32754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"91'410","reported_seq":2313,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144816+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:54:24.144816+0000","last_peered":"2024-09-14T22:54:24.144816+0000","last_clean":"2024-09-14T22:54:24.144816+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:54:24.144816+0000","last_undegraded":"2024-09-14T22:54:24.144816+0000","last_fullsized":"2024-09-14T22:54:24.144816+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:30:23.805939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":39407958,"num_objects":60,"num_object_clones":9,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":37,"num_read":3386,"num_read_kb":8895,"num_write":474,"num_write_kb":6537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"91'226","reported_seq":4268,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144105+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:54:24.144105+0000","last_peered":"2024-09-14T22:54:24.144105+0000","last_clean":"2024-09-14T22:54:24.144105+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:54:24.144105+0000","last_undegraded":"2024-09-14T22:54:24.144105+0000","last_fullsized":"2024-09-14T22:54:24.144105+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:39:51.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":76991017,"num_objects":68,"num_object_clones":9,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":41,"num_read":4028,"num_read_kb":50365,"num_write":231,"num_write_kb":17827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"91'335","reported_seq":11649,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148479+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:54:24.148479+0000","last_peered":"2024-09-14T22:54:24.148479+0000","last_clean":"2024-09-14T22:54:24.148479+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:54:24.148479+0000","last_undegraded":"2024-09-14T22:54:24.148479+0000","last_fullsized":"2024-09-14T22:54:24.148479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:06:27.925275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":59123683,"num_objects":95,"num_object_clones":24,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":53,"num_read":12125,"num_read_kb":48911,"num_write":277,"num_write_kb":28585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"90'293","reported_seq":15167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146232+0000","last_change":"2024-09-14T22:53:51.943155+0000","last_active":"2024-09-14T22:54:24.146232+0000","last_peered":"2024-09-14T22:54:24.146232+0000","last_clean":"2024-09-14T22:54:24.146232+0000","last_became_active":"2024-09-14T22:53:51.942663+0000","last_became_peered":"2024-09-14T22:53:51.942663+0000","last_unstale":"2024-09-14T22:54:24.146232+0000","last_undegraded":"2024-09-14T22:54:24.146232+0000","last_fullsized":"2024-09-14T22:54:24.146232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:42.672503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":75872371,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":58,"num_read":15236,"num_read_kb":91141,"num_write":260,"num_write_kb":37716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"unknown","last_fresh":"2024-09-14T22:54:24.145771+0000","last_change":"2024-09-14T22:54:24.819001+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:54:24.145771+0000","last_undegraded":"2024-09-14T22:54:24.145771+0000","last_fullsized":"2024-09-14T22:54:24.145771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"91'510","reported_seq":5967,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144867+0000","last_change":"2024-09-14T22:53:38.882316+0000","last_active":"2024-09-14T22:54:24.144867+0000","last_peered":"2024-09-14T22:54:24.144867+0000","last_clean":"2024-09-14T22:54:24.144867+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:54:24.144867+0000","last_undegraded":"2024-09-14T22:54:24.144867+0000","last_fullsized":"2024-09-14T22:54:24.144867+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:34.403347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":34010799,"num_objects":62,"num_object_clones":8,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":34,"num_read":7296,"num_read_kb":33358,"num_write":716,"num_write_kb":27062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"91'297","reported_seq":8684,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144852+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:54:24.144852+0000","last_peered":"2024-09-14T22:54:24.144852+0000","last_clean":"2024-09-14T22:54:24.144852+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:54:24.144852+0000","last_undegraded":"2024-09-14T22:54:24.144852+0000","last_fullsized":"2024-09-14T22:54:24.144852+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:50:15.160158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":61201000,"num_objects":77,"num_object_clones":17,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":48,"num_read":9460,"num_read_kb":43214,"num_write":257,"num_write_kb":15493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'211","reported_seq":8536,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146549+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:54:24.146549+0000","last_peered":"2024-09-14T22:54:24.146549+0000","last_clean":"2024-09-14T22:54:24.146549+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:54:24.146549+0000","last_undegraded":"2024-09-14T22:54:24.146549+0000","last_fullsized":"2024-09-14T22:54:24.146549+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:50.097468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":34741299,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":42,"num_read":8949,"num_read_kb":41372,"num_write":163,"num_write_kb":13237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"91'400","reported_seq":10339,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148343+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:54:24.148343+0000","last_peered":"2024-09-14T22:54:24.148343+0000","last_clean":"2024-09-14T22:54:24.148343+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:54:24.148343+0000","last_undegraded":"2024-09-14T22:54:24.148343+0000","last_fullsized":"2024-09-14T22:54:24.148343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:42.548136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":94275451,"num_objects":91,"num_object_clones":18,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":49,"num_read":10944,"num_read_kb":65418,"num_write":468,"num_write_kb":19420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143228+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:54:24.143228+0000","last_peered":"2024-09-14T22:54:24.143228+0000","last_clean":"2024-09-14T22:54:24.143228+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:54:24.143228+0000","last_undegraded":"2024-09-14T22:54:24.143228+0000","last_fullsized":"2024-09-14T22:54:24.143228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"91'507","reported_seq":7116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144555+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:54:24.144555+0000","last_peered":"2024-09-14T22:54:24.144555+0000","last_clean":"2024-09-14T22:54:24.144555+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:54:24.144555+0000","last_undegraded":"2024-09-14T22:54:24.144555+0000","last_fullsized":"2024-09-14T22:54:24.144555+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:17:07.382895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":73946266,"num_objects":69,"num_object_clones":11,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":39,"num_read":8733,"num_read_kb":39019,"num_write":604,"num_write_kb":23567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144961+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:54:24.144961+0000","last_peered":"2024-09-14T22:54:24.144961+0000","last_clean":"2024-09-14T22:54:24.144961+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:54:24.144961+0000","last_undegraded":"2024-09-14T22:54:24.144961+0000","last_fullsized":"2024-09-14T22:54:24.144961+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"91'194","reported_seq":1264,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146746+0000","last_change":"2024-09-14T22:54:07.044085+0000","last_active":"2024-09-14T22:54:24.146746+0000","last_peered":"2024-09-14T22:54:24.146746+0000","last_clean":"2024-09-14T22:54:24.146746+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:54:24.146746+0000","last_undegraded":"2024-09-14T22:54:24.146746+0000","last_fullsized":"2024-09-14T22:54:24.146746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:45:02.350894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":37076310,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":41,"num_read":885,"num_read_kb":11514,"num_write":183,"num_write_kb":14876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146646+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:54:24.146646+0000","last_peered":"2024-09-14T22:54:24.146646+0000","last_clean":"2024-09-14T22:54:24.146646+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:54:24.146646+0000","last_undegraded":"2024-09-14T22:54:24.146646+0000","last_fullsized":"2024-09-14T22:54:24.146646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"91'528","reported_seq":12864,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.147130+0000","last_change":"2024-09-14T22:53:51.942879+0000","last_active":"2024-09-14T22:54:24.147130+0000","last_peered":"2024-09-14T22:54:24.147130+0000","last_clean":"2024-09-14T22:54:24.147130+0000","last_became_active":"2024-09-14T22:53:51.942267+0000","last_became_peered":"2024-09-14T22:53:51.942267+0000","last_unstale":"2024-09-14T22:54:24.147130+0000","last_undegraded":"2024-09-14T22:54:24.147130+0000","last_fullsized":"2024-09-14T22:54:24.147130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:49.720399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":33598550,"num_objects":89,"num_object_clones":15,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":54,"num_read":14203,"num_read_kb":38154,"num_write":674,"num_write_kb":19221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143569+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:54:24.143569+0000","last_peered":"2024-09-14T22:54:24.143569+0000","last_clean":"2024-09-14T22:54:24.143569+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:54:24.143569+0000","last_undegraded":"2024-09-14T22:54:24.143569+0000","last_fullsized":"2024-09-14T22:54:24.143569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"91'303","reported_seq":14311,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.206854+0000","last_change":"2024-09-14T22:54:24.145705+0000","last_active":"2024-09-14T22:54:24.145705+0000","last_peered":"2024-09-14T22:54:23.206854+0000","last_clean":"2024-09-14T22:54:23.206854+0000","last_became_active":"2024-09-14T22:53:35.868554+0000","last_became_peered":"2024-09-14T22:53:35.868554+0000","last_unstale":"2024-09-14T22:54:23.206854+0000","last_undegraded":"2024-09-14T22:54:23.206854+0000","last_fullsized":"2024-09-14T22:54:23.206854+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":74854667,"num_objects":86,"num_object_clones":15,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":50,"num_read":13533,"num_read_kb":47864,"num_write":276,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":3794613,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"91'353","reported_seq":5121,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148539+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:54:24.148539+0000","last_peered":"2024-09-14T22:54:24.148539+0000","last_clean":"2024-09-14T22:54:24.148539+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:54:24.148539+0000","last_undegraded":"2024-09-14T22:54:24.148539+0000","last_fullsized":"2024-09-14T22:54:24.148539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:22:37.321498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":53351379,"num_objects":89,"num_object_clones":17,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":57,"num_read":6535,"num_read_kb":53355,"num_write":314,"num_write_kb":30002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144716+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:54:24.144716+0000","last_peered":"2024-09-14T22:54:24.144716+0000","last_clean":"2024-09-14T22:54:24.144716+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:54:24.144716+0000","last_undegraded":"2024-09-14T22:54:24.144716+0000","last_fullsized":"2024-09-14T22:54:24.144716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"91'223","reported_seq":1004,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143161+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:54:24.143161+0000","last_peered":"2024-09-14T22:54:24.143161+0000","last_clean":"2024-09-14T22:54:24.143161+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:54:24.143161+0000","last_undegraded":"2024-09-14T22:54:24.143161+0000","last_fullsized":"2024-09-14T22:54:24.143161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:34.116593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":12575758,"num_objects":58,"num_object_clones":8,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":981,"num_read_kb":7915,"num_write":230,"num_write_kb":11010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.187759+0000","last_change":"2024-09-14T22:54:24.146834+0000","last_active":"2024-09-14T22:54:24.146834+0000","last_peered":"2024-09-14T22:54:23.187759+0000","last_clean":"2024-09-14T22:54:23.187759+0000","last_became_active":"2024-09-14T22:53:32.745120+0000","last_became_peered":"2024-09-14T22:53:32.745120+0000","last_unstale":"2024-09-14T22:54:23.187759+0000","last_undegraded":"2024-09-14T22:54:23.187759+0000","last_fullsized":"2024-09-14T22:54:23.187759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"91'458","reported_seq":8405,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144171+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:54:24.144171+0000","last_peered":"2024-09-14T22:54:24.144171+0000","last_clean":"2024-09-14T22:54:24.144171+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:54:24.144171+0000","last_undegraded":"2024-09-14T22:54:24.144171+0000","last_fullsized":"2024-09-14T22:54:24.144171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:51:32.597165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":26518013,"num_objects":73,"num_object_clones":11,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":43,"num_read":9613,"num_read_kb":20323,"num_write":595,"num_write_kb":10180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146386+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:54:24.146386+0000","last_peered":"2024-09-14T22:54:24.146386+0000","last_clean":"2024-09-14T22:54:24.146386+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:54:24.146386+0000","last_undegraded":"2024-09-14T22:54:24.146386+0000","last_fullsized":"2024-09-14T22:54:24.146386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'259","reported_seq":5575,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148312+0000","last_change":"2024-09-14T22:54:18.745625+0000","last_active":"2024-09-14T22:54:24.148312+0000","last_peered":"2024-09-14T22:54:24.148312+0000","last_clean":"2024-09-14T22:54:24.148312+0000","last_became_active":"2024-09-14T22:54:18.744069+0000","last_became_peered":"2024-09-14T22:54:18.744069+0000","last_unstale":"2024-09-14T22:54:24.148312+0000","last_undegraded":"2024-09-14T22:54:24.148312+0000","last_fullsized":"2024-09-14T22:54:24.148312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:27.211239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":58509638,"num_objects":92,"num_object_clones":16,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":59,"num_read":4963,"num_read_kb":28758,"num_write":210,"num_write_kb":28454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":160,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144716+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:54:24.144716+0000","last_peered":"2024-09-14T22:54:24.144716+0000","last_clean":"2024-09-14T22:54:24.144716+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:54:24.144716+0000","last_undegraded":"2024-09-14T22:54:24.144716+0000","last_fullsized":"2024-09-14T22:54:24.144716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":310,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146453+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:54:24.146453+0000","last_peered":"2024-09-14T22:54:24.146453+0000","last_clean":"2024-09-14T22:54:24.146453+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:54:24.146453+0000","last_undegraded":"2024-09-14T22:54:24.146453+0000","last_fullsized":"2024-09-14T22:54:24.146453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"91'431","reported_seq":8221,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146404+0000","last_change":"2024-09-14T22:53:48.924907+0000","last_active":"2024-09-14T22:54:24.146404+0000","last_peered":"2024-09-14T22:54:24.146404+0000","last_clean":"2024-09-14T22:54:24.146404+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:54:24.146404+0000","last_undegraded":"2024-09-14T22:54:24.146404+0000","last_fullsized":"2024-09-14T22:54:24.146404+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:03:29.838215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":49431084,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":57,"num_read":10404,"num_read_kb":32163,"num_write":390,"num_write_kb":12535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"91'281","reported_seq":1287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148342+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:54:24.148342+0000","last_peered":"2024-09-14T22:54:24.148342+0000","last_clean":"2024-09-14T22:54:24.148342+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:54:24.148342+0000","last_undegraded":"2024-09-14T22:54:24.148342+0000","last_fullsized":"2024-09-14T22:54:24.148342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:45:15.048406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":22447246,"num_objects":84,"num_object_clones":15,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":57,"num_read":1795,"num_read_kb":6529,"num_write":246,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'345","reported_seq":1365,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143494+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:54:24.143494+0000","last_peered":"2024-09-14T22:54:24.143494+0000","last_clean":"2024-09-14T22:54:24.143494+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:54:24.143494+0000","last_undegraded":"2024-09-14T22:54:24.143494+0000","last_fullsized":"2024-09-14T22:54:24.143494+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:44.462443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":31221787,"num_objects":62,"num_object_clones":9,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":38,"num_read":2278,"num_read_kb":7206,"num_write":374,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"91'302","reported_seq":1934,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146253+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:54:24.146253+0000","last_peered":"2024-09-14T22:54:24.146253+0000","last_clean":"2024-09-14T22:54:24.146253+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:54:24.146253+0000","last_undegraded":"2024-09-14T22:54:24.146253+0000","last_fullsized":"2024-09-14T22:54:24.146253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:53:07.482998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":30215193,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":53,"num_read":2205,"num_read_kb":16804,"num_write":284,"num_write_kb":8479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"91'430","reported_seq":4684,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.325303+0000","last_change":"2024-09-14T22:54:24.145350+0000","last_active":"2024-09-14T22:54:24.145350+0000","last_peered":"2024-09-14T22:54:23.325303+0000","last_clean":"2024-09-14T22:54:23.325303+0000","last_became_active":"2024-09-14T22:53:35.868711+0000","last_became_peered":"2024-09-14T22:53:35.868711+0000","last_unstale":"2024-09-14T22:54:23.325303+0000","last_undegraded":"2024-09-14T22:54:23.325303+0000","last_fullsized":"2024-09-14T22:54:23.325303+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":32113789,"num_objects":79,"num_object_clones":9,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":49,"num_read":5574,"num_read_kb":20143,"num_write":458,"num_write_kb":16740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1667203,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"91'252","reported_seq":6115,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146644+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:54:24.146644+0000","last_peered":"2024-09-14T22:54:24.146644+0000","last_clean":"2024-09-14T22:54:24.146644+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:54:24.146644+0000","last_undegraded":"2024-09-14T22:54:24.146644+0000","last_fullsized":"2024-09-14T22:54:24.146644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:51.281278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":26665337,"num_objects":75,"num_object_clones":9,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":51,"num_read":6022,"num_read_kb":32214,"num_write":264,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"91'341","reported_seq":6296,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146744+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:54:24.146744+0000","last_peered":"2024-09-14T22:54:24.146744+0000","last_clean":"2024-09-14T22:54:24.146744+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:54:24.146744+0000","last_undegraded":"2024-09-14T22:54:24.146744+0000","last_fullsized":"2024-09-14T22:54:24.146744+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:40:33.245308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":56560750,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":47,"num_read":7166,"num_read_kb":39420,"num_write":329,"num_write_kb":20699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"91'228","reported_seq":5855,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146760+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:54:24.146760+0000","last_peered":"2024-09-14T22:54:24.146760+0000","last_clean":"2024-09-14T22:54:24.146760+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:54:24.146760+0000","last_undegraded":"2024-09-14T22:54:24.146760+0000","last_fullsized":"2024-09-14T22:54:24.146760+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:39.489218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":28566586,"num_objects":76,"num_object_clones":14,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":46,"num_read":5622,"num_read_kb":20719,"num_write":204,"num_write_kb":11550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"91'336","reported_seq":8846,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143312+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:54:24.143312+0000","last_peered":"2024-09-14T22:54:24.143312+0000","last_clean":"2024-09-14T22:54:24.143312+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:54:24.143312+0000","last_undegraded":"2024-09-14T22:54:24.143312+0000","last_fullsized":"2024-09-14T22:54:24.143312+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:47:33.886663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":68945901,"num_objects":77,"num_object_clones":11,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":46,"num_read":10314,"num_read_kb":28498,"num_write":320,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"90'439","reported_seq":4893,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146481+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:54:24.146481+0000","last_peered":"2024-09-14T22:54:24.146481+0000","last_clean":"2024-09-14T22:54:24.146481+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:54:24.146481+0000","last_undegraded":"2024-09-14T22:54:24.146481+0000","last_fullsized":"2024-09-14T22:54:24.146481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:31.691503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":54375844,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":41,"num_read":5815,"num_read_kb":32416,"num_write":575,"num_write_kb":22148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"91'309","reported_seq":1086,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146842+0000","last_change":"2024-09-14T22:53:38.883182+0000","last_active":"2024-09-14T22:54:24.146842+0000","last_peered":"2024-09-14T22:54:24.146842+0000","last_clean":"2024-09-14T22:54:24.146842+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:54:24.146842+0000","last_undegraded":"2024-09-14T22:54:24.146842+0000","last_fullsized":"2024-09-14T22:54:24.146842+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.546351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":47449396,"num_objects":71,"num_object_clones":11,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":50,"num_read":1181,"num_read_kb":9497,"num_write":358,"num_write_kb":9079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"92'762","reported_seq":5791,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146607+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:54:24.146607+0000","last_peered":"2024-09-14T22:54:24.146607+0000","last_clean":"2024-09-14T22:54:24.146607+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:54:24.146607+0000","last_undegraded":"2024-09-14T22:54:24.146607+0000","last_fullsized":"2024-09-14T22:54:24.146607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:11:57.358507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":56190029,"num_objects":84,"num_object_clones":13,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":56,"num_read":8125,"num_read_kb":8793,"num_write":988,"num_write_kb":3875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1451855270,"num_objects":2309,"num_object_clones":406,"num_object_copies":4618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2309,"num_whiteouts":1424,"num_read":214069,"num_read_kb":947727,"num_write":11567,"num_write_kb":504526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":34250838,"num_keys_recovered":18,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":683827200,"data_stored":799669840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10729,"ondisk_log_size":10729,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134648,"kb_used_data":90980,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94237192,"statfs":{"total":96636764160,"available":96498884608,"internally_reserved":0,"allocated":93163520,"data_stored":107519932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89832,"kb_used_data":54476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94282008,"statfs":{"total":96636764160,"available":96544776192,"internally_reserved":0,"allocated":55783424,"data_stored":63670191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121124,"kb_used_data":81568,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94250716,"statfs":{"total":96636764160,"available":96512733184,"internally_reserved":0,"allocated":83525632,"data_stored":92188032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206580,"kb_used_data":157340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94165260,"statfs":{"total":96636764160,"available":96425226240,"internally_reserved":0,"allocated":161116160,"data_stored":182895675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84732,"kb_used_data":49380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94287108,"statfs":{"total":96636764160,"available":96549998592,"internally_reserved":0,"allocated":50565120,"data_stored":55192059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79400,"kb_used_data":44044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94292440,"statfs":{"total":96636764160,"available":96555458560,"internally_reserved":0,"allocated":45101056,"data_stored":56158569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123052,"kb_used_data":87704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94248788,"statfs":{"total":96636764160,"available":96510758912,"internally_reserved":0,"allocated":89808896,"data_stored":116327143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157004,"kb_used_data":111932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94214836,"statfs":{"total":96636764160,"available":96475992064,"internally_reserved":0,"allocated":114618368,"data_stored":130909027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":112951296,"data_stored":129822208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88719360,"data_stored":115824272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":44015616,"data_stored":55655717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49479680,"data_stored":54689207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":160030720,"data_stored":182392823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":82436096,"data_stored":91685161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":54697984,"data_stored":63167339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":91496448,"data_stored":106433113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:54:28.956 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T22:54:28.959 INFO:teuthology.orchestra.run.smithi076.stdout:4709 punch from 0x150758a to 0x15104a7, (0x8f1d bytes) 2024-09-14T22:54:28.977 INFO:teuthology.orchestra.run.smithi076.stdout:4710 write 0xadecd9c thru 0xadf571c (0x8981 bytes) 2024-09-14T22:54:28.980 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.980 INFO:teuthology.orchestra.run.smithi076.stdout:4712 punch from 0x6cd1a73 to 0x6cdf06e, (0xd5fb bytes) 2024-09-14T22:54:28.980 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:28.990 INFO:teuthology.orchestra.run.smithi076.stdout:4714 write 0xe9de74c thru 0xe9e34c2 (0x4d77 bytes) 2024-09-14T22:54:28.994 INFO:teuthology.orchestra.run.smithi076.stdout:4715 writesame 0x6b0bf1b thru 0x6b187d5 data_size 0x7(0xc8bb bytes) 2024-09-14T22:54:29.001 INFO:teuthology.orchestra.run.smithi076.stdout:4716 read 0x659ca74 thru 0x65a992f (0xcebc bytes) 2024-09-14T22:54:29.002 INFO:teuthology.orchestra.run.smithi076.stdout:4718 read 0x29754aa thru 0x2984b05 (0xf65c bytes) 2024-09-14T22:54:29.014 INFO:teuthology.orchestra.run.smithi076.stdout:4719 trunc from 0xe9e34c3 to 0x49a1875 2024-09-14T22:54:29.027 INFO:teuthology.orchestra.run.smithi076.stdout:4720 write 0x14d051 thru 0x14e8c5 (0x1875 bytes) 2024-09-14T22:54:29.029 INFO:teuthology.orchestra.run.smithi076.stdout:4722 punch from 0x3a2c760 to 0x3a34de9, (0x8689 bytes) 2024-09-14T22:54:29.042 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:29.041+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:29.047 INFO:teuthology.orchestra.run.smithi076.stdout:4723 write 0xdf968b8 thru 0xdfa32a4 (0xc9ed bytes) 2024-09-14T22:54:29.050 INFO:teuthology.orchestra.run.smithi076.stdout:4726 punch from 0x99835c7 to 0x9991fb4, (0xe9ed bytes) 2024-09-14T22:54:29.051 INFO:teuthology.orchestra.run.smithi076.stdout:4727 punch from 0x8de3216 to 0x8de9be3, (0x69cd bytes) 2024-09-14T22:54:29.051 INFO:teuthology.orchestra.run.smithi076.stdout:4728 read 0x16da6e thru 0x176b93 (0x9126 bytes) 2024-09-14T22:54:29.051 INFO:teuthology.orchestra.run.smithi076.stdout:4729 write 0x6ca3222 thru 0x6cafa98 (0xc877 bytes) 2024-09-14T22:54:29.054 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:29.054 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:29.054 INFO:teuthology.orchestra.run.smithi076.stdout:4730 write 0x2f2863 thru 0x2f975a (0x6ef8 bytes) 2024-09-14T22:54:29.056 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.056 INFO:teuthology.orchestra.run.smithi076.stdout:4736 write 0x637acf thru 0x646e97 (0xf3c9 bytes) 2024-09-14T22:54:29.058 INFO:teuthology.orchestra.run.smithi076.stdout:4738 read 0x997abb4 thru 0x998a83a (0xfc87 bytes) 2024-09-14T22:54:29.065 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:29.072 INFO:teuthology.orchestra.run.smithi076.stdout:4739 trunc from 0xdfa32a5 to 0x2cdea91 2024-09-14T22:54:29.102 INFO:teuthology.orchestra.run.smithi076.stdout:4740 writesame 0xc9e6606 thru 0xc9f51dd data_size 0x4(0xebd8 bytes) 2024-09-14T22:54:29.103 INFO:teuthology.orchestra.run.smithi076.stdout:4741 write 0xab5a4ed thru 0xab67966 (0xd47a bytes) 2024-09-14T22:54:29.106 INFO:teuthology.orchestra.run.smithi076.stdout:4742 writesame 0x4651698 thru 0x465fcc7 data_size 0x1d(0xe630 bytes) 2024-09-14T22:54:29.116 INFO:teuthology.orchestra.run.smithi076.stdout:4743 trunc from 0xc9f51de to 0x6fc9094 2024-09-14T22:54:29.124 INFO:teuthology.orchestra.run.smithi076.stdout:4746 write 0x111baef thru 0x11232e6 (0x77f8 bytes) 2024-09-14T22:54:29.125 INFO:teuthology.orchestra.run.smithi076.stdout:4748 read 0xf913cd thru 0xf941ec (0x2e20 bytes) 2024-09-14T22:54:29.125 INFO:teuthology.orchestra.run.smithi076.stdout:4749 read 0x357b587 thru 0x357e69e (0x3118 bytes) 2024-09-14T22:54:29.127 INFO:teuthology.orchestra.run.smithi076.stdout:4750 trunc from 0x6fc9094 to 0x5e34eb0 2024-09-14T22:54:29.143 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:29.142+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:29.152 INFO:teuthology.orchestra.run.smithi076.stdout:4751 writesame 0xa415a9e thru 0xa421096 data_size 0x37(0xb5f9 bytes) 2024-09-14T22:54:29.152 INFO:teuthology.orchestra.run.smithi076.stdout:4752 punch from 0x6c85701 to 0x6c9154b, (0xbe4a bytes) 2024-09-14T22:54:29.152 INFO:teuthology.orchestra.run.smithi076.stdout:4753 read 0x3a3649f thru 0x3a4038f (0x9ef1 bytes) 2024-09-14T22:54:29.163 INFO:teuthology.orchestra.run.smithi076.stdout:4754 write 0xd3a1024 thru 0xd3a6fa6 (0x5f83 bytes) 2024-09-14T22:54:29.166 INFO:teuthology.orchestra.run.smithi076.stdout:4756 write 0x4e4ef3f thru 0x4e5561f (0x66e1 bytes) 2024-09-14T22:54:29.168 INFO:teuthology.orchestra.run.smithi076.stdout:4757 writesame 0x377d90b thru 0x37845c4 data_size 0x3(0x6cba bytes) 2024-09-14T22:54:29.173 INFO:teuthology.orchestra.run.smithi076.stdout:4759 punch from 0x15d3da to 0x16bbc7, (0xe7ed bytes) 2024-09-14T22:54:29.188 INFO:teuthology.orchestra.run.smithi076.stdout:4760 trunc from 0xd3a6fa7 to 0x24925bb 2024-09-14T22:54:29.190 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:29.190 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:29.200 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:29.214 INFO:teuthology.orchestra.run.smithi076.stdout:4762 write 0x77a5116 thru 0x77b2442 (0xd32d bytes) 2024-09-14T22:54:29.218 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.218 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.218 INFO:teuthology.orchestra.run.smithi076.stdout:4766 write 0x45addac thru 0x45bb38a (0xd5df bytes) 2024-09-14T22:54:29.219 INFO:teuthology.orchestra.run.smithi076.stdout:4767 punch from 0x5f2f153 to 0x5f30f70, (0x1e1d bytes) 2024-09-14T22:54:29.231 INFO:teuthology.orchestra.run.smithi076.stdout:4768 write 0xbf20b38 thru 0xbf210de (0x5a7 bytes) 2024-09-14T22:54:29.244 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:29.242+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:29.244 INFO:teuthology.orchestra.run.smithi076.stdout:4769 write 0xecedaf8 thru 0xecfaa9a (0xcfa3 bytes) 2024-09-14T22:54:29.247 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.247 INFO:teuthology.orchestra.run.smithi076.stdout:4773 read 0x50a34df thru 0x50a54bc (0x1fde bytes) 2024-09-14T22:54:29.247 INFO:teuthology.orchestra.run.smithi076.stdout:4775 trunc from 0xecfaa9b to 0xe952508 2024-09-14T22:54:29.255 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.255 INFO:teuthology.orchestra.run.smithi076.stdout:4777 read 0x2c42f9e thru 0x2c488c5 (0x5928 bytes) 2024-09-14T22:54:29.255 INFO:teuthology.orchestra.run.smithi076.stdout:4781 write 0x52ed780 thru 0x52fd31d (0xfb9e bytes) 2024-09-14T22:54:29.257 INFO:teuthology.orchestra.run.smithi076.stdout:4782 read 0x71952e8 thru 0x7195308 (0x21 bytes) 2024-09-14T22:54:29.257 INFO:teuthology.orchestra.run.smithi076.stdout:4783 writesame 0x4c22f69 thru 0x4c25846 data_size 0x1(0x28de bytes) 2024-09-14T22:54:29.266 INFO:teuthology.orchestra.run.smithi076.stdout:4784 read 0x282f9aa thru 0x283bd76 (0xc3cd bytes) 2024-09-14T22:54:29.267 INFO:teuthology.orchestra.run.smithi076.stdout:4785 write 0xdba22bd thru 0xdba5bbe (0x3902 bytes) 2024-09-14T22:54:29.270 INFO:teuthology.orchestra.run.smithi076.stdout:4787 punch from 0x68b60d to 0x698d9f, (0xd792 bytes) 2024-09-14T22:54:29.271 INFO:teuthology.orchestra.run.smithi076.stdout:4789 writesame 0x1589262 thru 0x158cff2 data_size 0x1(0x3d91 bytes) 2024-09-14T22:54:29.282 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.282 INFO:teuthology.orchestra.run.smithi076.stdout:4792 writesame 0x65853e6 thru 0x658ca73 data_size 0x19(0x768e bytes) 2024-09-14T22:54:29.284 INFO:teuthology.orchestra.run.smithi076.stdout:4793 punch from 0x1b3669d to 0x1b3cf00, (0x6863 bytes) 2024-09-14T22:54:29.285 INFO:teuthology.orchestra.run.smithi076.stdout:4794 read 0x7f0b1b2 thru 0x7f16ffc (0xbe4b bytes) 2024-09-14T22:54:29.285 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.285 INFO:teuthology.orchestra.run.smithi076.stdout:4797 write 0xd278454 thru 0xd284791 (0xc33e bytes) 2024-09-14T22:54:29.288 INFO:teuthology.orchestra.run.smithi076.stdout:4798 read 0xe91aedc thru 0xe923043 (0x8168 bytes) 2024-09-14T22:54:29.288 INFO:teuthology.orchestra.run.smithi076.stdout:4799 write 0x6915c35 thru 0x6921c3b (0xc007 bytes) 2024-09-14T22:54:29.296 INFO:teuthology.orchestra.run.smithi076.stdout:4800 trunc from 0xe952508 to 0xb56c167 2024-09-14T22:54:29.308 INFO:teuthology.orchestra.run.smithi076.stdout:4802 write 0x68fc24c thru 0x690b47f (0xf234 bytes) 2024-09-14T22:54:29.309 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.309 INFO:teuthology.orchestra.run.smithi076.stdout:4805 punch from 0x2208a60 to 0x22177a3, (0xed43 bytes) 2024-09-14T22:54:29.309 INFO:teuthology.orchestra.run.smithi076.stdout:4806 punch from 0x930f08f to 0x931dba4, (0xeb15 bytes) 2024-09-14T22:54:29.312 INFO:teuthology.orchestra.run.smithi076.stdout:4807 trunc from 0xb56c167 to 0xd7c3520 2024-09-14T22:54:29.317 INFO:teuthology.orchestra.run.smithi076.stdout:4808 write 0xce21606 thru 0xce27d7e (0x6779 bytes) 2024-09-14T22:54:29.320 INFO:teuthology.orchestra.run.smithi076.stdout:4809 punch from 0xcad63bd to 0xcae5585, (0xf1c8 bytes) 2024-09-14T22:54:29.320 INFO:teuthology.orchestra.run.smithi076.stdout:4810 write 0x74899fe thru 0x7499273 (0xf876 bytes) 2024-09-14T22:54:29.322 INFO:teuthology.orchestra.run.smithi076.stdout:4812 write 0x37c0a4c thru 0x37cb01e (0xa5d3 bytes) 2024-09-14T22:54:29.323 INFO:teuthology.orchestra.run.smithi076.stdout:4814 read 0x3582ded thru 0x358de17 (0xb02b bytes) 2024-09-14T22:54:29.324 INFO:teuthology.orchestra.run.smithi076.stdout:4816 punch from 0x49f9250 to 0x4a05452, (0xc202 bytes) 2024-09-14T22:54:29.324 INFO:teuthology.orchestra.run.smithi076.stdout:4817 read 0x2048fec thru 0x20585c3 (0xf5d8 bytes) 2024-09-14T22:54:29.325 INFO:teuthology.orchestra.run.smithi076.stdout:4818 read 0xc2ab6ef thru 0xc2b1538 (0x5e4a bytes) 2024-09-14T22:54:29.325 INFO:teuthology.orchestra.run.smithi076.stdout:4819 write 0x14215c9 thru 0x14217a1 (0x1d9 bytes) 2024-09-14T22:54:29.326 INFO:teuthology.orchestra.run.smithi076.stdout:4820 writesame 0xa5ec35b thru 0xa5f55c0 data_size 0x7(0x9266 bytes) 2024-09-14T22:54:29.336 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:29.336 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:29.344 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:29.343+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:29.350 INFO:teuthology.orchestra.run.smithi076.stdout:4821 trunc from 0xd7c3520 to 0x113d471 2024-09-14T22:54:29.357 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:29.363 INFO:teuthology.orchestra.run.smithi076.stdout:4824 punch from 0xfbd88 to 0x1041cd, (0x8445 bytes) 2024-09-14T22:54:29.363 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.379 INFO:teuthology.orchestra.run.smithi076.stdout:4827 trunc from 0x113d471 to 0xe874f37 2024-09-14T22:54:29.385 INFO:teuthology.orchestra.run.smithi076.stdout:4829 write 0xc2d3625 thru 0xc2e026d (0xcc49 bytes) 2024-09-14T22:54:29.386 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:29.386 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":111,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1726354369,"num_in_osds":8,"osd_in_since":1726354360,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":36},{"state_name":"peering","count":3},{"state_name":"unknown","count":2}],"num_pgs":41,"num_pools":3,"num_objects":2313,"data_bytes":1452445657,"bytes_used":1020284928,"bytes_avail":772073828352,"bytes_total":773094113280,"unknown_pgs_ratio":0.048780485987663269,"inactive_pgs_ratio":0.073170728981494904,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":283},"fsmap":{"epoch":1,"btime":"2024-09-14T22:52:37:937082+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-14T22:53:50.792085+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-14T22:54:29.388 INFO:teuthology.orchestra.run.smithi076.stdout:4830 read 0x8a472b8 thru 0x8a533d9 (0xc122 bytes) 2024-09-14T22:54:29.388 INFO:teuthology.orchestra.run.smithi076.stdout:4831 read 0xfff816 thru 0x100f395 (0xfb80 bytes) 2024-09-14T22:54:29.388 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.388 INFO:teuthology.orchestra.run.smithi076.stdout:4834 read 0x2ae0a8b thru 0x2ae5ac3 (0x5039 bytes) 2024-09-14T22:54:29.389 INFO:teuthology.orchestra.run.smithi076.stdout:4835 read 0x17b5e81 thru 0x17b6a58 (0xbd8 bytes) 2024-09-14T22:54:29.389 INFO:teuthology.orchestra.run.smithi076.stdout:4836 punch from 0xd23b008 to 0xd247614, (0xc60c bytes) 2024-09-14T22:54:29.389 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.389 INFO:teuthology.orchestra.run.smithi076.stdout:4839 read 0x38c1dd6 thru 0x38cce1a (0xb045 bytes) 2024-09-14T22:54:29.389 INFO:teuthology.orchestra.run.smithi076.stdout:4840 write 0x66623de thru 0x666ad6e (0x8991 bytes) 2024-09-14T22:54:29.393 INFO:teuthology.orchestra.run.smithi076.stdout:4841 write 0xe7069e1 thru 0xe715234 (0xe854 bytes) 2024-09-14T22:54:29.396 INFO:teuthology.orchestra.run.smithi076.stdout:4842 writesame 0x1c71d11 thru 0x1c80f5b data_size 0x1(0xf24b bytes) 2024-09-14T22:54:29.401 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-14T22:54:29.401 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:54:29.448 INFO:teuthology.orchestra.run.smithi076.stdout:4843 trunc from 0xe874f37 to 0x6d93b4f 2024-09-14T22:54:29.449 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:29.447+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:29.463 INFO:teuthology.orchestra.run.smithi076.stdout:4846 writesame 0x4e27fcc thru 0x4e31b0e data_size 0x1(0x9b43 bytes) 2024-09-14T22:54:29.488 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:29.488 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:29.494 INFO:teuthology.orchestra.run.smithi076.stdout:4847 trunc from 0x6d93b4f to 0xb7d203f 2024-09-14T22:54:29.501 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:29.502 INFO:teuthology.orchestra.run.smithi076.stdout:4848 punch from 0x121f781 to 0x1228c10, (0x948f bytes) 2024-09-14T22:54:29.502 INFO:teuthology.orchestra.run.smithi076.stdout:4850 trunc from 0xb7d203f to 0xc123dfe 2024-09-14T22:54:29.518 INFO:teuthology.orchestra.run.smithi076.stdout:4852 write 0xdfc8238 thru 0xdfd1ea6 (0x9c6f bytes) 2024-09-14T22:54:29.533 INFO:teuthology.orchestra.run.smithi076.stdout:4853 trunc from 0xdfd1ea7 to 0x4dc93c6 2024-09-14T22:54:29.546 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:29.544+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:29.548 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:29.555 INFO:teuthology.orchestra.run.smithi076.stdout:4855 write 0x5661fa9 thru 0x566cb4c (0xaba4 bytes) 2024-09-14T22:54:29.556 INFO:teuthology.orchestra.run.smithi076.stdout:4856 trunc from 0x566cb4d to 0x530241e 2024-09-14T22:54:29.582 INFO:teuthology.orchestra.run.smithi076.stdout:4860 write 0xe49421e thru 0xe49e2ed (0xa0d0 bytes) 2024-09-14T22:54:29.585 INFO:teuthology.orchestra.run.smithi076.stdout:4862 write 0xd89bc46 thru 0xd89cf82 (0x133d bytes) 2024-09-14T22:54:29.592 INFO:teuthology.orchestra.run.smithi076.stdout:4863 trunc from 0xe49e2ee to 0x9de1ffe 2024-09-14T22:54:29.602 INFO:teuthology.orchestra.run.smithi076.stdout:4864 write 0xe97738 thru 0xe9d182 (0x5a4b bytes) 2024-09-14T22:54:29.604 INFO:teuthology.orchestra.run.smithi076.stdout:4865 read 0x4d9b64c thru 0x4da8e9f (0xd854 bytes) 2024-09-14T22:54:29.604 INFO:teuthology.orchestra.run.smithi076.stdout:4866 clone 38 order 21 su 524288 sc 5 2024-09-14T22:54:29.625 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:29.625 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:29.634 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:29.647 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:29.646+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:29.747 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:29.746+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:29.749 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:29.750 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:54:29.761 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-14T22:54:28.798505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1452445657,"num_objects":2313,"num_object_clones":406,"num_object_copies":4626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2313,"num_whiteouts":1424,"num_read":214185,"num_read_kb":947827,"num_write":11728,"num_write_kb":506944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":34250838,"num_keys_recovered":18,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10849,"ondisk_log_size":10849,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":996372,"kb_used_data":677424,"kb_used_omap":12,"kb_used_meta":318707,"kb_avail":753978348,"statfs":{"total":773094113280,"available":772073828352,"internally_reserved":0,"allocated":693682176,"data_stored":804860628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":326356529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-31770909,"num_objects":60,"num_object_clones":30,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":2,"num_read":-1817,"num_read_kb":-3225,"num_write":1136,"num_write_kb":-7803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-98,"num_bytes_recovered":-1352199,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000625"},"pg_stats":[{"pgid":"3.1f","version":"91'490","reported_seq":7543,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143791+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:54:24.143791+0000","last_peered":"2024-09-14T22:54:24.143791+0000","last_clean":"2024-09-14T22:54:24.143791+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:54:24.143791+0000","last_undegraded":"2024-09-14T22:54:24.143791+0000","last_fullsized":"2024-09-14T22:54:24.143791+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:21.625600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":45640341,"num_objects":79,"num_object_clones":17,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":47,"num_read":8754,"num_read_kb":28886,"num_write":579,"num_write_kb":6883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"unknown","last_fresh":"2024-09-14T22:54:24.145826+0000","last_change":"2024-09-14T22:54:24.819044+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:54:24.145826+0000","last_undegraded":"2024-09-14T22:54:24.145826+0000","last_fullsized":"2024-09-14T22:54:24.145826+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"90'276","reported_seq":7781,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.145100+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:54:24.145100+0000","last_peered":"2024-09-14T22:54:24.145100+0000","last_clean":"2024-09-14T22:54:24.145100+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:54:24.145100+0000","last_undegraded":"2024-09-14T22:54:24.145100+0000","last_fullsized":"2024-09-14T22:54:24.145100+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":276,"log_dups_size":0,"ondisk_log_size":276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:06:32.916398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":55977828,"num_objects":80,"num_object_clones":16,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":47,"num_read":7944,"num_read_kb":34863,"num_write":266,"num_write_kb":32754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"91'410","reported_seq":2313,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144816+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:54:24.144816+0000","last_peered":"2024-09-14T22:54:24.144816+0000","last_clean":"2024-09-14T22:54:24.144816+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:54:24.144816+0000","last_undegraded":"2024-09-14T22:54:24.144816+0000","last_fullsized":"2024-09-14T22:54:24.144816+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:30:23.805939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":39407958,"num_objects":60,"num_object_clones":9,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":37,"num_read":3386,"num_read_kb":8895,"num_write":474,"num_write_kb":6537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"91'226","reported_seq":4268,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144105+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:54:24.144105+0000","last_peered":"2024-09-14T22:54:24.144105+0000","last_clean":"2024-09-14T22:54:24.144105+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:54:24.144105+0000","last_undegraded":"2024-09-14T22:54:24.144105+0000","last_fullsized":"2024-09-14T22:54:24.144105+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:39:51.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":76991017,"num_objects":68,"num_object_clones":9,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":41,"num_read":4028,"num_read_kb":50365,"num_write":231,"num_write_kb":17827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"91'335","reported_seq":11649,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148479+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:54:24.148479+0000","last_peered":"2024-09-14T22:54:24.148479+0000","last_clean":"2024-09-14T22:54:24.148479+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:54:24.148479+0000","last_undegraded":"2024-09-14T22:54:24.148479+0000","last_fullsized":"2024-09-14T22:54:24.148479+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:06:27.925275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":59123683,"num_objects":95,"num_object_clones":24,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":53,"num_read":12125,"num_read_kb":48911,"num_write":277,"num_write_kb":28585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"90'293","reported_seq":15167,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146232+0000","last_change":"2024-09-14T22:53:51.943155+0000","last_active":"2024-09-14T22:54:24.146232+0000","last_peered":"2024-09-14T22:54:24.146232+0000","last_clean":"2024-09-14T22:54:24.146232+0000","last_became_active":"2024-09-14T22:53:51.942663+0000","last_became_peered":"2024-09-14T22:53:51.942663+0000","last_unstale":"2024-09-14T22:54:24.146232+0000","last_undegraded":"2024-09-14T22:54:24.146232+0000","last_fullsized":"2024-09-14T22:54:24.146232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:42.672503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":75872371,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":58,"num_read":15236,"num_read_kb":91141,"num_write":260,"num_write_kb":37716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"unknown","last_fresh":"2024-09-14T22:54:24.145771+0000","last_change":"2024-09-14T22:54:24.819001+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T22:54:24.145771+0000","last_undegraded":"2024-09-14T22:54:24.145771+0000","last_fullsized":"2024-09-14T22:54:24.145771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"91'510","reported_seq":5967,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144867+0000","last_change":"2024-09-14T22:53:38.882316+0000","last_active":"2024-09-14T22:54:24.144867+0000","last_peered":"2024-09-14T22:54:24.144867+0000","last_clean":"2024-09-14T22:54:24.144867+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:54:24.144867+0000","last_undegraded":"2024-09-14T22:54:24.144867+0000","last_fullsized":"2024-09-14T22:54:24.144867+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:34.403347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":34010799,"num_objects":62,"num_object_clones":8,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":34,"num_read":7296,"num_read_kb":33358,"num_write":716,"num_write_kb":27062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"91'297","reported_seq":8684,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144852+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:54:24.144852+0000","last_peered":"2024-09-14T22:54:24.144852+0000","last_clean":"2024-09-14T22:54:24.144852+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:54:24.144852+0000","last_undegraded":"2024-09-14T22:54:24.144852+0000","last_fullsized":"2024-09-14T22:54:24.144852+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:50:15.160158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":61201000,"num_objects":77,"num_object_clones":17,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":48,"num_read":9460,"num_read_kb":43214,"num_write":257,"num_write_kb":15493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'211","reported_seq":8536,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146549+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:54:24.146549+0000","last_peered":"2024-09-14T22:54:24.146549+0000","last_clean":"2024-09-14T22:54:24.146549+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:54:24.146549+0000","last_undegraded":"2024-09-14T22:54:24.146549+0000","last_fullsized":"2024-09-14T22:54:24.146549+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:50.097468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":34741299,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":42,"num_read":8949,"num_read_kb":41372,"num_write":163,"num_write_kb":13237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"91'400","reported_seq":10339,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148343+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:54:24.148343+0000","last_peered":"2024-09-14T22:54:24.148343+0000","last_clean":"2024-09-14T22:54:24.148343+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:54:24.148343+0000","last_undegraded":"2024-09-14T22:54:24.148343+0000","last_fullsized":"2024-09-14T22:54:24.148343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:42.548136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":94275451,"num_objects":91,"num_object_clones":18,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":49,"num_read":10944,"num_read_kb":65418,"num_write":468,"num_write_kb":19420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143228+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:54:24.143228+0000","last_peered":"2024-09-14T22:54:24.143228+0000","last_clean":"2024-09-14T22:54:24.143228+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:54:24.143228+0000","last_undegraded":"2024-09-14T22:54:24.143228+0000","last_fullsized":"2024-09-14T22:54:24.143228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"91'507","reported_seq":7116,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144555+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:54:24.144555+0000","last_peered":"2024-09-14T22:54:24.144555+0000","last_clean":"2024-09-14T22:54:24.144555+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:54:24.144555+0000","last_undegraded":"2024-09-14T22:54:24.144555+0000","last_fullsized":"2024-09-14T22:54:24.144555+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:17:07.382895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":73946266,"num_objects":69,"num_object_clones":11,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":39,"num_read":8733,"num_read_kb":39019,"num_write":604,"num_write_kb":23567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":118,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144961+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:54:24.144961+0000","last_peered":"2024-09-14T22:54:24.144961+0000","last_clean":"2024-09-14T22:54:24.144961+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:54:24.144961+0000","last_undegraded":"2024-09-14T22:54:24.144961+0000","last_fullsized":"2024-09-14T22:54:24.144961+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"91'194","reported_seq":1264,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146746+0000","last_change":"2024-09-14T22:54:07.044085+0000","last_active":"2024-09-14T22:54:24.146746+0000","last_peered":"2024-09-14T22:54:24.146746+0000","last_clean":"2024-09-14T22:54:24.146746+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:54:24.146746+0000","last_undegraded":"2024-09-14T22:54:24.146746+0000","last_fullsized":"2024-09-14T22:54:24.146746+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:45:02.350894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":37076310,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":41,"num_read":885,"num_read_kb":11514,"num_write":183,"num_write_kb":14876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146646+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:54:24.146646+0000","last_peered":"2024-09-14T22:54:24.146646+0000","last_clean":"2024-09-14T22:54:24.146646+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:54:24.146646+0000","last_undegraded":"2024-09-14T22:54:24.146646+0000","last_fullsized":"2024-09-14T22:54:24.146646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"91'528","reported_seq":12864,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.147130+0000","last_change":"2024-09-14T22:53:51.942879+0000","last_active":"2024-09-14T22:54:24.147130+0000","last_peered":"2024-09-14T22:54:24.147130+0000","last_clean":"2024-09-14T22:54:24.147130+0000","last_became_active":"2024-09-14T22:53:51.942267+0000","last_became_peered":"2024-09-14T22:53:51.942267+0000","last_unstale":"2024-09-14T22:54:24.147130+0000","last_undegraded":"2024-09-14T22:54:24.147130+0000","last_fullsized":"2024-09-14T22:54:24.147130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:49.720399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":33598550,"num_objects":89,"num_object_clones":15,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":54,"num_read":14203,"num_read_kb":38154,"num_write":674,"num_write_kb":19221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143569+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:54:24.143569+0000","last_peered":"2024-09-14T22:54:24.143569+0000","last_clean":"2024-09-14T22:54:24.143569+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:54:24.143569+0000","last_undegraded":"2024-09-14T22:54:24.143569+0000","last_fullsized":"2024-09-14T22:54:24.143569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"91'303","reported_seq":14311,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.206854+0000","last_change":"2024-09-14T22:54:24.145705+0000","last_active":"2024-09-14T22:54:24.145705+0000","last_peered":"2024-09-14T22:54:23.206854+0000","last_clean":"2024-09-14T22:54:23.206854+0000","last_became_active":"2024-09-14T22:53:35.868554+0000","last_became_peered":"2024-09-14T22:53:35.868554+0000","last_unstale":"2024-09-14T22:54:23.206854+0000","last_undegraded":"2024-09-14T22:54:23.206854+0000","last_fullsized":"2024-09-14T22:54:23.206854+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":74854667,"num_objects":86,"num_object_clones":15,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":50,"num_read":13533,"num_read_kb":47864,"num_write":276,"num_write_kb":18600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":3794613,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"91'353","reported_seq":5121,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148539+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:54:24.148539+0000","last_peered":"2024-09-14T22:54:24.148539+0000","last_clean":"2024-09-14T22:54:24.148539+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:54:24.148539+0000","last_undegraded":"2024-09-14T22:54:24.148539+0000","last_fullsized":"2024-09-14T22:54:24.148539+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:22:37.321498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":53351379,"num_objects":89,"num_object_clones":17,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":57,"num_read":6535,"num_read_kb":53355,"num_write":314,"num_write_kb":30002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144716+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:54:24.144716+0000","last_peered":"2024-09-14T22:54:24.144716+0000","last_clean":"2024-09-14T22:54:24.144716+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:54:24.144716+0000","last_undegraded":"2024-09-14T22:54:24.144716+0000","last_fullsized":"2024-09-14T22:54:24.144716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"91'223","reported_seq":1004,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143161+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:54:24.143161+0000","last_peered":"2024-09-14T22:54:24.143161+0000","last_clean":"2024-09-14T22:54:24.143161+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:54:24.143161+0000","last_undegraded":"2024-09-14T22:54:24.143161+0000","last_fullsized":"2024-09-14T22:54:24.143161+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:34.116593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":12575758,"num_objects":58,"num_object_clones":8,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":34,"num_read":981,"num_read_kb":7915,"num_write":230,"num_write_kb":11010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.187759+0000","last_change":"2024-09-14T22:54:24.146834+0000","last_active":"2024-09-14T22:54:24.146834+0000","last_peered":"2024-09-14T22:54:23.187759+0000","last_clean":"2024-09-14T22:54:23.187759+0000","last_became_active":"2024-09-14T22:53:32.745120+0000","last_became_peered":"2024-09-14T22:53:32.745120+0000","last_unstale":"2024-09-14T22:54:23.187759+0000","last_undegraded":"2024-09-14T22:54:23.187759+0000","last_fullsized":"2024-09-14T22:54:23.187759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"91'458","reported_seq":8405,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144171+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:54:24.144171+0000","last_peered":"2024-09-14T22:54:24.144171+0000","last_clean":"2024-09-14T22:54:24.144171+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:54:24.144171+0000","last_undegraded":"2024-09-14T22:54:24.144171+0000","last_fullsized":"2024-09-14T22:54:24.144171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:51:32.597165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":26518013,"num_objects":73,"num_object_clones":11,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":43,"num_read":9613,"num_read_kb":20323,"num_write":595,"num_write_kb":10180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146386+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:54:24.146386+0000","last_peered":"2024-09-14T22:54:24.146386+0000","last_clean":"2024-09-14T22:54:24.146386+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:54:24.146386+0000","last_undegraded":"2024-09-14T22:54:24.146386+0000","last_fullsized":"2024-09-14T22:54:24.146386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'259","reported_seq":5575,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148312+0000","last_change":"2024-09-14T22:54:18.745625+0000","last_active":"2024-09-14T22:54:24.148312+0000","last_peered":"2024-09-14T22:54:24.148312+0000","last_clean":"2024-09-14T22:54:24.148312+0000","last_became_active":"2024-09-14T22:54:18.744069+0000","last_became_peered":"2024-09-14T22:54:18.744069+0000","last_unstale":"2024-09-14T22:54:24.148312+0000","last_undegraded":"2024-09-14T22:54:24.148312+0000","last_fullsized":"2024-09-14T22:54:24.148312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:27.211239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":58509638,"num_objects":92,"num_object_clones":16,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":59,"num_read":4963,"num_read_kb":28758,"num_write":210,"num_write_kb":28454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":160,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.144716+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:54:24.144716+0000","last_peered":"2024-09-14T22:54:24.144716+0000","last_clean":"2024-09-14T22:54:24.144716+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:54:24.144716+0000","last_undegraded":"2024-09-14T22:54:24.144716+0000","last_fullsized":"2024-09-14T22:54:24.144716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":310,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146453+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:54:24.146453+0000","last_peered":"2024-09-14T22:54:24.146453+0000","last_clean":"2024-09-14T22:54:24.146453+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:54:24.146453+0000","last_undegraded":"2024-09-14T22:54:24.146453+0000","last_fullsized":"2024-09-14T22:54:24.146453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"91'431","reported_seq":8221,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146404+0000","last_change":"2024-09-14T22:53:48.924907+0000","last_active":"2024-09-14T22:54:24.146404+0000","last_peered":"2024-09-14T22:54:24.146404+0000","last_clean":"2024-09-14T22:54:24.146404+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:54:24.146404+0000","last_undegraded":"2024-09-14T22:54:24.146404+0000","last_fullsized":"2024-09-14T22:54:24.146404+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:03:29.838215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":49431084,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":57,"num_read":10404,"num_read_kb":32163,"num_write":390,"num_write_kb":12535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"91'281","reported_seq":1287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.148342+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:54:24.148342+0000","last_peered":"2024-09-14T22:54:24.148342+0000","last_clean":"2024-09-14T22:54:24.148342+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:54:24.148342+0000","last_undegraded":"2024-09-14T22:54:24.148342+0000","last_fullsized":"2024-09-14T22:54:24.148342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:45:15.048406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":22447246,"num_objects":84,"num_object_clones":15,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":57,"num_read":1795,"num_read_kb":6529,"num_write":246,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'345","reported_seq":1365,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143494+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:54:24.143494+0000","last_peered":"2024-09-14T22:54:24.143494+0000","last_clean":"2024-09-14T22:54:24.143494+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:54:24.143494+0000","last_undegraded":"2024-09-14T22:54:24.143494+0000","last_fullsized":"2024-09-14T22:54:24.143494+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:44.462443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":31221787,"num_objects":62,"num_object_clones":9,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":38,"num_read":2278,"num_read_kb":7206,"num_write":374,"num_write_kb":4107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"91'302","reported_seq":1934,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146253+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:54:24.146253+0000","last_peered":"2024-09-14T22:54:24.146253+0000","last_clean":"2024-09-14T22:54:24.146253+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:54:24.146253+0000","last_undegraded":"2024-09-14T22:54:24.146253+0000","last_fullsized":"2024-09-14T22:54:24.146253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:53:07.482998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":30215193,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":53,"num_read":2205,"num_read_kb":16804,"num_write":284,"num_write_kb":8479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"91'430","reported_seq":4684,"reported_epoch":91,"state":"peering","last_fresh":"2024-09-14T22:54:23.325303+0000","last_change":"2024-09-14T22:54:24.145350+0000","last_active":"2024-09-14T22:54:24.145350+0000","last_peered":"2024-09-14T22:54:23.325303+0000","last_clean":"2024-09-14T22:54:23.325303+0000","last_became_active":"2024-09-14T22:53:35.868711+0000","last_became_peered":"2024-09-14T22:53:35.868711+0000","last_unstale":"2024-09-14T22:54:23.325303+0000","last_undegraded":"2024-09-14T22:54:23.325303+0000","last_fullsized":"2024-09-14T22:54:23.325303+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":32113789,"num_objects":79,"num_object_clones":9,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":49,"num_read":5574,"num_read_kb":20143,"num_write":458,"num_write_kb":16740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":1667203,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"91'252","reported_seq":6115,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146644+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:54:24.146644+0000","last_peered":"2024-09-14T22:54:24.146644+0000","last_clean":"2024-09-14T22:54:24.146644+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:54:24.146644+0000","last_undegraded":"2024-09-14T22:54:24.146644+0000","last_fullsized":"2024-09-14T22:54:24.146644+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:51.281278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":26665337,"num_objects":75,"num_object_clones":9,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":51,"num_read":6022,"num_read_kb":32214,"num_write":264,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"91'341","reported_seq":6296,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146744+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:54:24.146744+0000","last_peered":"2024-09-14T22:54:24.146744+0000","last_clean":"2024-09-14T22:54:24.146744+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:54:24.146744+0000","last_undegraded":"2024-09-14T22:54:24.146744+0000","last_fullsized":"2024-09-14T22:54:24.146744+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:40:33.245308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":56560750,"num_objects":73,"num_object_clones":15,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":47,"num_read":7166,"num_read_kb":39420,"num_write":329,"num_write_kb":20699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"91'228","reported_seq":5855,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146760+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:54:24.146760+0000","last_peered":"2024-09-14T22:54:24.146760+0000","last_clean":"2024-09-14T22:54:24.146760+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:54:24.146760+0000","last_undegraded":"2024-09-14T22:54:24.146760+0000","last_fullsized":"2024-09-14T22:54:24.146760+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:39.489218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":28566586,"num_objects":76,"num_object_clones":14,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":46,"num_read":5622,"num_read_kb":20719,"num_write":204,"num_write_kb":11550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"91'336","reported_seq":8846,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.143312+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:54:24.143312+0000","last_peered":"2024-09-14T22:54:24.143312+0000","last_clean":"2024-09-14T22:54:24.143312+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:54:24.143312+0000","last_undegraded":"2024-09-14T22:54:24.143312+0000","last_fullsized":"2024-09-14T22:54:24.143312+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:47:33.886663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":68945901,"num_objects":77,"num_object_clones":11,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":46,"num_read":10314,"num_read_kb":28498,"num_write":320,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"90'439","reported_seq":4893,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146481+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:54:24.146481+0000","last_peered":"2024-09-14T22:54:24.146481+0000","last_clean":"2024-09-14T22:54:24.146481+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:54:24.146481+0000","last_undegraded":"2024-09-14T22:54:24.146481+0000","last_fullsized":"2024-09-14T22:54:24.146481+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:31.691503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":54375844,"num_objects":71,"num_object_clones":15,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":41,"num_read":5815,"num_read_kb":32416,"num_write":575,"num_write_kb":22148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"91'309","reported_seq":1086,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146842+0000","last_change":"2024-09-14T22:53:38.883182+0000","last_active":"2024-09-14T22:54:24.146842+0000","last_peered":"2024-09-14T22:54:24.146842+0000","last_clean":"2024-09-14T22:54:24.146842+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:54:24.146842+0000","last_undegraded":"2024-09-14T22:54:24.146842+0000","last_fullsized":"2024-09-14T22:54:24.146842+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.546351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":47449396,"num_objects":71,"num_object_clones":11,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":50,"num_read":1181,"num_read_kb":9497,"num_write":358,"num_write_kb":9079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"92'762","reported_seq":5791,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T22:54:24.146607+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:54:24.146607+0000","last_peered":"2024-09-14T22:54:24.146607+0000","last_clean":"2024-09-14T22:54:24.146607+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:54:24.146607+0000","last_undegraded":"2024-09-14T22:54:24.146607+0000","last_fullsized":"2024-09-14T22:54:24.146607+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:11:57.358507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":56190029,"num_objects":84,"num_object_clones":13,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":56,"num_read":8125,"num_read_kb":8793,"num_write":988,"num_write_kb":3875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1451855270,"num_objects":2309,"num_object_clones":406,"num_object_copies":4618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2309,"num_whiteouts":1424,"num_read":214069,"num_read_kb":947727,"num_write":11567,"num_write_kb":504526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":34250838,"num_keys_recovered":18,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":683827200,"data_stored":799669840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10729,"ondisk_log_size":10729,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134648,"kb_used_data":90980,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94237192,"statfs":{"total":96636764160,"available":96498884608,"internally_reserved":0,"allocated":93163520,"data_stored":107519932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89832,"kb_used_data":54476,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94282008,"statfs":{"total":96636764160,"available":96544776192,"internally_reserved":0,"allocated":55783424,"data_stored":63670191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542167,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121124,"kb_used_data":81568,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94250716,"statfs":{"total":96636764160,"available":96512733184,"internally_reserved":0,"allocated":83525632,"data_stored":92188032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542167,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206580,"kb_used_data":157340,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94165260,"statfs":{"total":96636764160,"available":96425226240,"internally_reserved":0,"allocated":161116160,"data_stored":182895675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84732,"kb_used_data":49380,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94287108,"statfs":{"total":96636764160,"available":96549998592,"internally_reserved":0,"allocated":50565120,"data_stored":55192059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79400,"kb_used_data":44044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94292440,"statfs":{"total":96636764160,"available":96555458560,"internally_reserved":0,"allocated":45101056,"data_stored":56158569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123052,"kb_used_data":87704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94248788,"statfs":{"total":96636764160,"available":96510758912,"internally_reserved":0,"allocated":89808896,"data_stored":116327143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157004,"kb_used_data":111932,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94214836,"statfs":{"total":96636764160,"available":96475992064,"internally_reserved":0,"allocated":114618368,"data_stored":130909027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":112951296,"data_stored":129822208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88719360,"data_stored":115824272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":44015616,"data_stored":55655717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49479680,"data_stored":54689207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":160030720,"data_stored":182392823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":82436096,"data_stored":91685161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":54697984,"data_stored":63167339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":91496448,"data_stored":106433113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:54:29.780 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:29.780 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:29.790 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:29.875 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:29.874+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:29.934 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:29.934 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:29.944 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:29.991 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:29.989+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:30.051 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:30.050+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:30.094 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:30.094 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:30.103 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:30.152 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:30.151+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:30.193 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:54:30.211 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:30.211 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:30.220 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:30.264 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:30.263+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:30.323 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:30.323 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:30.331 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:30.354 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:30.353+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:30.407 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-14T22:54:30.442 INFO:teuthology.orchestra.run.smithi076.stdout:4867 read 0x7a7cb4c thru 0x7a8bb6f (0xf024 bytes) 2024-09-14T22:54:30.443 INFO:teuthology.orchestra.run.smithi076.stdout:4868 read 0x59f376b thru 0x59fa72a (0x6fc0 bytes) 2024-09-14T22:54:30.444 INFO:teuthology.orchestra.run.smithi076.stdout:4869 write 0x9b9629c thru 0x9ba022a (0x9f8f bytes) 2024-09-14T22:54:30.451 INFO:teuthology.orchestra.run.smithi076.stdout:4870 trunc from 0x9de1ffe to 0x9846287 2024-09-14T22:54:30.454 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:30.453+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:30.462 INFO:teuthology.orchestra.run.smithi076.stdout:4871 write 0x2929d85 thru 0x2933a4d (0x9cc9 bytes) 2024-09-14T22:54:30.465 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.471 INFO:teuthology.orchestra.run.smithi076.stdout:4873 write 0xaedc415 thru 0xaee36d2 (0x72be bytes) 2024-09-14T22:54:30.474 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.474 INFO:teuthology.orchestra.run.smithi076.stdout:4875 read 0x1286109 thru 0x12904a6 (0xa39e bytes) 2024-09-14T22:54:30.475 INFO:teuthology.orchestra.run.smithi076.stdout:4876 write 0x8aec83e thru 0x8af0f43 (0x4706 bytes) 2024-09-14T22:54:30.478 INFO:teuthology.orchestra.run.smithi076.stdout:4879 writesame 0x25a93fe thru 0x25b0fbd data_size 0x63(0x7bc0 bytes) 2024-09-14T22:54:30.482 INFO:teuthology.orchestra.run.smithi076.stdout:4881 punch from 0x6d22f9 to 0x6d74b1, (0x51b8 bytes) 2024-09-14T22:54:30.482 INFO:teuthology.orchestra.run.smithi076.stdout:4882 read 0xa5ea58a thru 0xa5f6d7f (0xc7f6 bytes) 2024-09-14T22:54:30.482 INFO:teuthology.orchestra.run.smithi076.stdout:4883 read 0x2ffdc47 thru 0x300af37 (0xd2f1 bytes) 2024-09-14T22:54:30.483 INFO:teuthology.orchestra.run.smithi076.stdout:4884 read 0xae7adfd thru 0xae80fd3 (0x61d7 bytes) 2024-09-14T22:54:30.483 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.483 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.483 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.484 INFO:teuthology.orchestra.run.smithi076.stdout:4891 writesame 0x1a29147 thru 0x1a32499 data_size 0x13(0x9353 bytes) 2024-09-14T22:54:30.487 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:30.487 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:30.488 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.489 INFO:teuthology.orchestra.run.smithi076.stdout:4894 trunc from 0xaee36d3 to 0x951e830 2024-09-14T22:54:30.497 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:30.503 INFO:teuthology.orchestra.run.smithi076.stdout:4895 read 0x3155fcb thru 0x315f088 (0x90be bytes) 2024-09-14T22:54:30.503 INFO:teuthology.orchestra.run.smithi076.stdout:4896 read 0xa0acf5 thru 0xa153e9 (0xa6f5 bytes) 2024-09-14T22:54:30.508 INFO:teuthology.orchestra.run.smithi076.stdout:4897 write 0x9737c24 thru 0x973b29c (0x3679 bytes) 2024-09-14T22:54:30.511 INFO:teuthology.orchestra.run.smithi076.stdout:4898 read 0x60b1617 thru 0x60bf5c3 (0xdfad bytes) 2024-09-14T22:54:30.511 INFO:teuthology.orchestra.run.smithi076.stdout:4900 write 0x6e4ef33 thru 0x6e57f2c (0x8ffa bytes) 2024-09-14T22:54:30.514 INFO:teuthology.orchestra.run.smithi076.stdout:4901 read 0x75f9a9f thru 0x7600496 (0x69f8 bytes) 2024-09-14T22:54:30.514 INFO:teuthology.orchestra.run.smithi076.stdout:4902 writesame 0x4fa7de4 thru 0x4fb378c data_size 0x3(0xb9a9 bytes) 2024-09-14T22:54:30.526 INFO:teuthology.orchestra.run.smithi076.stdout:4903 write 0x9367ca1 thru 0x937703f (0xf39f bytes) 2024-09-14T22:54:30.529 INFO:teuthology.orchestra.run.smithi076.stdout:4904 punch from 0x6e20b48 to 0x6e2ecec, (0xe1a4 bytes) 2024-09-14T22:54:30.529 INFO:teuthology.orchestra.run.smithi076.stdout:4905 read 0x28de84 thru 0x29584a (0x79c7 bytes) 2024-09-14T22:54:30.529 INFO:teuthology.orchestra.run.smithi076.stdout:4906 write 0x61c9b06 thru 0x61d72ee (0xd7e9 bytes) 2024-09-14T22:54:30.533 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.534 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.534 INFO:teuthology.orchestra.run.smithi076.stdout:4911 read 0x7464c59 thru 0x746bd95 (0x713d bytes) 2024-09-14T22:54:30.534 INFO:teuthology.orchestra.run.smithi076.stdout:4912 read 0x48ce8df thru 0x48ceaf8 (0x21a bytes) 2024-09-14T22:54:30.534 INFO:teuthology.orchestra.run.smithi076.stdout:4913 read 0x3c4786a thru 0x3c4b769 (0x3f00 bytes) 2024-09-14T22:54:30.535 INFO:teuthology.orchestra.run.smithi076.stdout:4915 write 0x3370d17 thru 0x337e6e2 (0xd9cc bytes) 2024-09-14T22:54:30.537 INFO:teuthology.orchestra.run.smithi076.stdout:4917 read 0x3cca2bb thru 0x3cca898 (0x5de bytes) 2024-09-14T22:54:30.543 INFO:teuthology.orchestra.run.smithi076.stdout:4918 trunc from 0x973b29d to 0xdc8e0e5 2024-09-14T22:54:30.548 INFO:teuthology.orchestra.run.smithi076.stdout:4919 read 0x3673412 thru 0x367807d (0x4c6c bytes) 2024-09-14T22:54:30.549 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.549 INFO:teuthology.orchestra.run.smithi076.stdout:4921 punch from 0x5d936af to 0x5d97cf5, (0x4646 bytes) 2024-09-14T22:54:30.549 INFO:teuthology.orchestra.run.smithi076.stdout:4922 write 0x9a4b9b3 thru 0x9a51fc5 (0x6613 bytes) 2024-09-14T22:54:30.552 INFO:teuthology.orchestra.run.smithi076.stdout:4923 writesame 0x159d07b thru 0x15a87b7 data_size 0x1(0xb73d bytes) 2024-09-14T22:54:30.555 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:30.554+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:30.577 INFO:teuthology.orchestra.run.smithi076.stdout:4924 write 0x990422c thru 0x990bcbd (0x7a92 bytes) 2024-09-14T22:54:30.582 INFO:teuthology.orchestra.run.smithi076.stdout:4926 writesame 0xcf49e12 thru 0xcf58597 data_size 0x5(0xe786 bytes) 2024-09-14T22:54:30.591 INFO:teuthology.orchestra.run.smithi076.stdout:4927 writesame 0xd6dbadd thru 0xd6debd8 data_size 0xc(0x30fc bytes) 2024-09-14T22:54:30.597 INFO:teuthology.orchestra.run.smithi076.stdout:4928 trunc from 0xdc8e0e5 to 0xeb56503 2024-09-14T22:54:30.604 INFO:teuthology.orchestra.run.smithi076.stdout:4930 writesame 0x554415c thru 0x5548fb7 data_size 0x55(0x4e5c bytes) 2024-09-14T22:54:30.608 INFO:teuthology.orchestra.run.smithi076.stdout:4931 read 0xbaef1e6 thru 0xbafa60b (0xb426 bytes) 2024-09-14T22:54:30.608 INFO:teuthology.orchestra.run.smithi076.stdout:4933 read 0x18d85df thru 0x18d9f2d (0x194f bytes) 2024-09-14T22:54:30.609 INFO:teuthology.orchestra.run.smithi076.stdout:4934 writesame 0x9a397ec thru 0x9a4285a data_size 0x4b(0x906f bytes) 2024-09-14T22:54:30.610 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.611 INFO:teuthology.orchestra.run.smithi076.stdout:4936 write 0xd87f36f thru 0xd888d16 (0x99a8 bytes) 2024-09-14T22:54:30.616 INFO:teuthology.orchestra.run.smithi076.stdout:4937 write 0xbda2195 thru 0xbdb0d8d (0xebf9 bytes) 2024-09-14T22:54:30.621 INFO:teuthology.orchestra.run.smithi076.stdout:4938 write 0x58e9ff thru 0x59d18d (0xe78f bytes) 2024-09-14T22:54:30.627 INFO:teuthology.orchestra.run.smithi076.stdout:4940 write 0x39b42cc thru 0x39bcafd (0x8832 bytes) 2024-09-14T22:54:30.631 INFO:teuthology.orchestra.run.smithi076.stdout:4941 read 0xbf77adf thru 0xbf85043 (0xd565 bytes) 2024-09-14T22:54:30.631 INFO:teuthology.orchestra.run.smithi076.stdout:4942 write 0xcc2c990 thru 0xcc3ba76 (0xf0e7 bytes) 2024-09-14T22:54:30.634 INFO:teuthology.orchestra.run.smithi076.stdout:4943 write 0xd25a677 thru 0xd26281f (0x81a9 bytes) 2024-09-14T22:54:30.637 INFO:teuthology.orchestra.run.smithi076.stdout:4945 write 0x7ca6eae thru 0x7caab01 (0x3c54 bytes) 2024-09-14T22:54:30.644 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:30.645 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:30.649 INFO:teuthology.orchestra.run.smithi076.stdout:4946 trunc from 0xeb56503 to 0x7bc6996 2024-09-14T22:54:30.653 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:30.655 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:30.654+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:30.672 INFO:teuthology.orchestra.run.smithi076.stdout:4947 punch from 0x510ebb9 to 0x5110ad6, (0x1f1d bytes) 2024-09-14T22:54:30.673 INFO:teuthology.orchestra.run.smithi076.stdout:4948 punch from 0x40b7221 to 0x40c57a3, (0xe582 bytes) 2024-09-14T22:54:30.673 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.683 INFO:teuthology.orchestra.run.smithi076.stdout:4952 write 0xaecaa9c thru 0xaed7ed4 (0xd439 bytes) 2024-09-14T22:54:30.686 INFO:teuthology.orchestra.run.smithi076.stdout:4954 read 0x2e048dd thru 0x2e12ac2 (0xe1e6 bytes) 2024-09-14T22:54:30.694 INFO:teuthology.orchestra.run.smithi076.stdout:4955 writesame 0xd40b3bb thru 0xd416f66 data_size 0x2(0xbbac bytes) 2024-09-14T22:54:30.699 INFO:teuthology.orchestra.run.smithi076.stdout:4956 trunc from 0xd416f67 to 0x88602b0 2024-09-14T22:54:30.709 INFO:teuthology.orchestra.run.smithi076.stdout:4957 read 0x2d36b27 thru 0x2d3da22 (0x6efc bytes) 2024-09-14T22:54:30.709 INFO:teuthology.orchestra.run.smithi076.stdout:4959 write 0x37aaf6d thru 0x37ac2a5 (0x1339 bytes) 2024-09-14T22:54:30.713 INFO:teuthology.orchestra.run.smithi076.stdout:4961 writesame 0x494cd6d thru 0x4952394 data_size 0xc(0x5628 bytes) 2024-09-14T22:54:30.717 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.717 INFO:teuthology.orchestra.run.smithi076.stdout:4965 read 0x822372a thru 0x822e688 (0xaf5f bytes) 2024-09-14T22:54:30.717 INFO:teuthology.orchestra.run.smithi076.stdout:4966 write 0x750b87 thru 0x75df10 (0xd38a bytes) 2024-09-14T22:54:30.728 INFO:teuthology.orchestra.run.smithi076.stdout:4967 trunc from 0x88602b0 to 0x3502d24 2024-09-14T22:54:30.752 INFO:teuthology.orchestra.run.smithi076.stdout:4968 write 0x527d3e9 thru 0x5285314 (0x7f2c bytes) 2024-09-14T22:54:30.757 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:30.756+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:30.757 INFO:teuthology.orchestra.run.smithi076.stdout:4969 punch from 0x4a3dcf3 to 0x4a4a06e, (0xc37b bytes) 2024-09-14T22:54:30.763 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:30.763 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:30.765 INFO:teuthology.orchestra.run.smithi076.stdout:4971 write 0x7327e80 thru 0x7333dc3 (0xbf44 bytes) 2024-09-14T22:54:30.768 INFO:teuthology.orchestra.run.smithi076.stdout:4972 read 0x33c063d thru 0x33d032c (0xfcf0 bytes) 2024-09-14T22:54:30.769 INFO:teuthology.orchestra.run.smithi076.stdout:4974 write 0x456e7db thru 0x456f65b (0xe81 bytes) 2024-09-14T22:54:30.772 INFO:teuthology.orchestra.run.smithi076.stdout:4975 writesame 0x17cf926 thru 0x17d15a8 data_size 0x3(0x1c83 bytes) 2024-09-14T22:54:30.773 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:30.775 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.775 INFO:teuthology.orchestra.run.smithi076.stdout:4978 punch from 0x34e9a8e to 0x34ec8ee, (0x2e60 bytes) 2024-09-14T22:54:30.775 INFO:teuthology.orchestra.run.smithi076.stdout:4979 punch from 0x6984977 to 0x698c08f, (0x7718 bytes) 2024-09-14T22:54:30.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:30.789 INFO:teuthology.orchestra.run.smithi076.stdout:4980 write 0xe9e0972 thru 0xe9ef733 (0xedc2 bytes) 2024-09-14T22:54:30.792 INFO:teuthology.orchestra.run.smithi076.stdout:4981 writesame 0x5bd08fa thru 0x5bdbf7f data_size 0x3d(0xb686 bytes) 2024-09-14T22:54:30.796 INFO:teuthology.orchestra.run.smithi076.stdout:4982 read 0x5db16c4 thru 0x5db4765 (0x30a2 bytes) 2024-09-14T22:54:30.796 INFO:teuthology.orchestra.run.smithi076.stdout:4983 writesame 0xb37938e thru 0xb37d0b3 data_size 0x3(0x3d26 bytes) 2024-09-14T22:54:30.802 INFO:teuthology.orchestra.run.smithi076.stdout:4985 writesame 0x5f73526 thru 0x5f7db9c data_size 0xf(0xa677 bytes) 2024-09-14T22:54:30.806 INFO:teuthology.orchestra.run.smithi076.stdout:4986 write 0x8ce7297 thru 0x8cf03b2 (0x911c bytes) 2024-09-14T22:54:30.809 INFO:teuthology.orchestra.run.smithi076.stdout:4988 read 0xa9a000d thru 0xa9a3479 (0x346d bytes) 2024-09-14T22:54:30.810 INFO:teuthology.orchestra.run.smithi076.stdout:4990 read 0x4d4fa60 thru 0x4d5e003 (0xe5a4 bytes) 2024-09-14T22:54:30.810 INFO:teuthology.orchestra.run.smithi076.stdout:4991 write 0x46bcfa4 thru 0x46ccb65 (0xfbc2 bytes) 2024-09-14T22:54:30.813 INFO:teuthology.orchestra.run.smithi076.stdout:4992 punch from 0x66dad1b to 0x66df392, (0x4677 bytes) 2024-09-14T22:54:30.813 INFO:teuthology.orchestra.run.smithi076.stdout:4993 writesame 0x43ddcd thru 0x4448c8 data_size 0x4(0x6afc bytes) 2024-09-14T22:54:30.823 INFO:teuthology.orchestra.run.smithi076.stdout:4994 read 0x36b1cdd thru 0x36b5963 (0x3c87 bytes) 2024-09-14T22:54:30.823 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.823 INFO:teuthology.orchestra.run.smithi076.stdout:4996 punch from 0x932bea2 to 0x932eb7b, (0x2cd9 bytes) 2024-09-14T22:54:30.833 INFO:teuthology.orchestra.run.smithi076.stdout:4997 trunc from 0xe9ef734 to 0x6c71033 2024-09-14T22:54:30.846 INFO:teuthology.orchestra.run.smithi076.stdout:4998 trunc from 0x6c71033 to 0x40f8350 2024-09-14T22:54:30.857 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:30.856+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:30.860 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.860 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.862 INFO:teuthology.orchestra.run.smithi076.stdout:5001 trunc from 0x40f8350 to 0x2d8f408 2024-09-14T22:54:30.881 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:30.881 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:30.883 INFO:teuthology.orchestra.run.smithi076.stdout:5002 writesame 0x2ac23ab thru 0x2ac817e data_size 0xa(0x5dd4 bytes) 2024-09-14T22:54:30.887 INFO:teuthology.orchestra.run.smithi076.stdout:5003 read 0x22ce0d2 thru 0x22db9c9 (0xd8f8 bytes) 2024-09-14T22:54:30.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:30.902 INFO:teuthology.orchestra.run.smithi076.stdout:5004 write 0xa65e787 thru 0xa66d650 (0xeeca bytes) 2024-09-14T22:54:30.909 INFO:teuthology.orchestra.run.smithi076.stdout:5005 trunc from 0xa66d651 to 0x7614ffe 2024-09-14T22:54:30.913 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:30.914 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:30.932 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:30.932 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:30.932 INFO:teuthology.orchestra.run.smithi076.stdout:5006 writesame 0xc089f89 thru 0xc08cd4c data_size 0x3a(0x2dc4 bytes) 2024-09-14T22:54:30.933 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.933 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:30.933 INFO:teuthology.orchestra.run.smithi076.stdout:5013 writesame 0xa5ff3a thru 0xa6990d data_size 0x2c(0x99d4 bytes) 2024-09-14T22:54:30.936 INFO:teuthology.orchestra.run.smithi076.stdout:5014 writesame 0x4639b4a thru 0x4646fe5 data_size 0x16(0xd49c bytes) 2024-09-14T22:54:30.951 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:30.951 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:30.955 INFO:teuthology.orchestra.run.smithi076.stdout:5015 trunc from 0xc08cd4d to 0x149b2be 2024-09-14T22:54:30.970 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:30.971 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:30.975 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:30.974+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:30.977 INFO:teuthology.orchestra.run.smithi076.stdout:5017 read 0x8a2e77 thru 0x8ab516 (0x86a0 bytes) 2024-09-14T22:54:30.980 INFO:teuthology.orchestra.run.smithi076.stdout:5018 trunc from 0x149b2be to 0x3f127f1 2024-09-14T22:54:30.988 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:30.988 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:30.989 INFO:teuthology.orchestra.run.smithi076.stdout:5019 punch from 0xccbd75 to 0xcd0113, (0x439e bytes) 2024-09-14T22:54:30.994 INFO:teuthology.orchestra.run.smithi076.stdout:5020 trunc from 0x3f127f1 to 0x8b354f5 2024-09-14T22:54:31.004 INFO:teuthology.orchestra.run.smithi076.stdout:5021 trunc from 0x8b354f5 to 0x5cdf9e7 2024-09-14T22:54:31.006 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:31.009 INFO:teuthology.orchestra.run.smithi076.stdout:5022 punch from 0x49ed6ca to 0x49fa5f9, (0xcf2f bytes) 2024-09-14T22:54:31.009 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.009 INFO:teuthology.orchestra.run.smithi076.stdout:5024 punch from 0x1f5be19 to 0x1f60dca, (0x4fb1 bytes) 2024-09-14T22:54:31.017 INFO:teuthology.orchestra.run.smithi076.stdout:5025 writesame 0x7268b63 thru 0x72723d1 data_size 0x1(0x986f bytes) 2024-09-14T22:54:31.018 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.018 INFO:teuthology.orchestra.run.smithi076.stdout:5027 read 0x1b1439d thru 0x1b18672 (0x42d6 bytes) 2024-09-14T22:54:31.018 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.018 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.018 INFO:teuthology.orchestra.run.smithi076.stdout:5032 punch from 0x21df6bd to 0x21eeef2, (0xf835 bytes) 2024-09-14T22:54:31.023 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:31.023 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:31.028 INFO:teuthology.orchestra.run.smithi076.stdout:5033 writesame 0xc85da2b thru 0xc863601 data_size 0x3(0x5bd7 bytes) 2024-09-14T22:54:31.028 INFO:teuthology.orchestra.run.smithi076.stdout:5034 read 0x67e63a7 thru 0x67f3bee (0xd848 bytes) 2024-09-14T22:54:31.028 INFO:teuthology.orchestra.run.smithi076.stdout:5035 read 0x9fa3ac8 thru 0x9fa41c2 (0x6fb bytes) 2024-09-14T22:54:31.028 INFO:teuthology.orchestra.run.smithi076.stdout:5036 write 0x58d57f0 thru 0x58e08d0 (0xb0e1 bytes) 2024-09-14T22:54:31.031 INFO:teuthology.orchestra.run.smithi076.stdout:5038 writesame 0xa884e10 thru 0xa890dfd data_size 0x39(0xbfee bytes) 2024-09-14T22:54:31.034 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.034 INFO:teuthology.orchestra.run.smithi076.stdout:5040 write 0x39d5508 thru 0x39e3c78 (0xe771 bytes) 2024-09-14T22:54:31.037 INFO:teuthology.orchestra.run.smithi076.stdout:5041 writesame 0x63d7bd8 thru 0x63e548d data_size 0x1(0xd8b6 bytes) 2024-09-14T22:54:31.039 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:31.039 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:31.040 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:31.040 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:31.042 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:31.042 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:31.049 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:31.068 INFO:teuthology.orchestra.run.smithi076.stdout:5042 punch from 0x746a633 to 0x746f353, (0x4d20 bytes) 2024-09-14T22:54:31.068 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.068 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.068 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.068 INFO:teuthology.orchestra.run.smithi076.stdout:5050 read 0xbb8a3d4 thru 0xbb92467 (0x8094 bytes) 2024-09-14T22:54:31.069 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:31.067+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:31.076 INFO:teuthology.orchestra.run.smithi076.stdout:5051 write 0xcd679f8 thru 0xcd736ae (0xbcb7 bytes) 2024-09-14T22:54:31.079 INFO:teuthology.orchestra.run.smithi076.stdout:5052 read 0x105deec thru 0x105ff05 (0x201a bytes) 2024-09-14T22:54:31.086 INFO:teuthology.orchestra.run.smithi076.stdout:5053 write 0xe39a189 thru 0xe39a4ad (0x325 bytes) 2024-09-14T22:54:31.089 INFO:teuthology.orchestra.run.smithi076.stdout:5054 punch from 0x4feee74 to 0x4ff1039, (0x21c5 bytes) 2024-09-14T22:54:31.090 INFO:teuthology.orchestra.run.smithi076.stdout:5055 punch from 0x3624bf3 to 0x362ca29, (0x7e36 bytes) 2024-09-14T22:54:31.099 INFO:teuthology.orchestra.run.smithi076.stdout:5056 trunc from 0xe39a4ae to 0x5584137 2024-09-14T22:54:31.112 INFO:teuthology.orchestra.run.smithi076.stdout:5058 read 0xda6b8c thru 0xdb48a3 (0xdd18 bytes) 2024-09-14T22:54:31.112 INFO:teuthology.orchestra.run.smithi076.stdout:5059 read 0x273d17 thru 0x27ac72 (0x6f5c bytes) 2024-09-14T22:54:31.119 INFO:teuthology.orchestra.run.smithi076.stdout:5060 write 0x667c2b3 thru 0x66825e8 (0x6336 bytes) 2024-09-14T22:54:31.123 INFO:teuthology.orchestra.run.smithi076.stdout:5061 writesame 0x454485b thru 0x454a98e data_size 0x2(0x6134 bytes) 2024-09-14T22:54:31.132 INFO:teuthology.orchestra.run.smithi076.stdout:5062 write 0x52f34a5 thru 0x5301e1a (0xe976 bytes) 2024-09-14T22:54:31.136 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.136 INFO:teuthology.orchestra.run.smithi076.stdout:5064 write 0x3558c7a thru 0x356573c (0xcac3 bytes) 2024-09-14T22:54:31.147 INFO:teuthology.orchestra.run.smithi076.stdout:5066 write 0x8ceb2b1 thru 0x8cedba9 (0x28f9 bytes) 2024-09-14T22:54:31.150 INFO:teuthology.orchestra.run.smithi076.stdout:5070 write 0x3412ec2 thru 0x341c4d7 (0x9616 bytes) 2024-09-14T22:54:31.153 INFO:teuthology.orchestra.run.smithi076.stdout:5071 read 0x649c2cd thru 0x64a933c (0xd070 bytes) 2024-09-14T22:54:31.153 INFO:teuthology.orchestra.run.smithi076.stdout:5072 writesame 0x3936743 thru 0x393cb3b data_size 0x13(0x63f9 bytes) 2024-09-14T22:54:31.161 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:31.160+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:31.168 INFO:teuthology.orchestra.run.smithi076.stdout:5076 writesame 0xe5da536 thru 0xe5e707e data_size 0x53(0xcb49 bytes) 2024-09-14T22:54:31.169 INFO:teuthology.orchestra.run.smithi076.stdout:5077 read 0x5c85f8 thru 0x5cf39c (0x6da5 bytes) 2024-09-14T22:54:31.169 INFO:teuthology.orchestra.run.smithi076.stdout:5078 punch from 0xa749171 to 0xa756b49, (0xd9d8 bytes) 2024-09-14T22:54:31.169 INFO:teuthology.orchestra.run.smithi076.stdout:5079 writesame 0xeeae9c thru 0xef2bbe data_size 0x2b(0x7d23 bytes) 2024-09-14T22:54:31.174 INFO:teuthology.orchestra.run.smithi076.stdout:5080 write 0xd2bad5d thru 0xd2c2d81 (0x8025 bytes) 2024-09-14T22:54:31.177 INFO:teuthology.orchestra.run.smithi076.stdout:5082 read 0x41b9654 thru 0x41c2720 (0x90cd bytes) 2024-09-14T22:54:31.177 INFO:teuthology.orchestra.run.smithi076.stdout:5083 writesame 0x5eb1131 thru 0x5eb3d74 data_size 0x2(0x2c44 bytes) 2024-09-14T22:54:31.182 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.185 INFO:teuthology.orchestra.run.smithi076.stdout:5088 trunc from 0xe5e707f to 0xc2f4a31 2024-09-14T22:54:31.196 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:54:31.198 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:31.198 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:31.199 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:31.199 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:31.199 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:31.208 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:31.208 INFO:teuthology.orchestra.run.smithi076.stdout:5091 trunc from 0xc2f4a31 to 0x44a388d 2024-09-14T22:54:31.225 INFO:teuthology.orchestra.run.smithi076.stdout:5093 trunc from 0x44a388d to 0x75e1a81 2024-09-14T22:54:31.232 INFO:teuthology.orchestra.run.smithi076.stdout:5095 read 0x9f7004 thru 0x9fb463 (0x4460 bytes) 2024-09-14T22:54:31.233 INFO:teuthology.orchestra.run.smithi076.stdout:5096 trunc from 0x75e1a81 to 0x747f7e8 2024-09-14T22:54:31.237 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.244 INFO:teuthology.orchestra.run.smithi076.stdout:5099 write 0x8644026 thru 0x8648eac (0x4e87 bytes) 2024-09-14T22:54:31.247 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.248 INFO:teuthology.orchestra.run.smithi076.stdout:5101 punch from 0x3fb3a69 to 0x3fbc957, (0x8eee bytes) 2024-09-14T22:54:31.248 INFO:teuthology.orchestra.run.smithi076.stdout:5102 read 0x18d3f20 thru 0x18debaf (0xac90 bytes) 2024-09-14T22:54:31.258 INFO:teuthology.orchestra.run.smithi076.stdout:5103 write 0xc4c068a thru 0xc4cc8de (0xc255 bytes) 2024-09-14T22:54:31.261 INFO:teuthology.orchestra.run.smithi076.stdout:5104 write 0xae69776 thru 0xae6a793 (0x101e bytes) 2024-09-14T22:54:31.262 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:31.260+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:31.265 INFO:teuthology.orchestra.run.smithi076.stdout:5105 write 0xa7179ef thru 0xa727696 (0xfca8 bytes) 2024-09-14T22:54:31.272 INFO:teuthology.orchestra.run.smithi076.stdout:5106 trunc from 0xc4cc8df to 0x8c962d6 2024-09-14T22:54:31.284 INFO:teuthology.orchestra.run.smithi076.stdout:5107 read 0x7b77067 thru 0x7b841c2 (0xd15c bytes) 2024-09-14T22:54:31.284 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.284 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.284 INFO:teuthology.orchestra.run.smithi076.stdout:5110 writesame 0x78a90b8 thru 0x78a9779 data_size 0x5(0x6c2 bytes) 2024-09-14T22:54:31.286 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.296 INFO:teuthology.orchestra.run.smithi076.stdout:5114 write 0xae569d3 thru 0xae63ac2 (0xd0f0 bytes) 2024-09-14T22:54:31.298 INFO:teuthology.orchestra.run.smithi076.stdout:5115 write 0x49c0c22 thru 0x49c0eca (0x2a9 bytes) 2024-09-14T22:54:31.302 INFO:teuthology.orchestra.run.smithi076.stdout:5116 read 0x2c449e0 thru 0x2c4a7b2 (0x5dd3 bytes) 2024-09-14T22:54:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:5117 punch from 0x2660625 to 0x266da16, (0xd3f1 bytes) 2024-09-14T22:54:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:5118 read 0x12e5c1b thru 0x12ecd85 (0x716b bytes) 2024-09-14T22:54:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:5119 punch from 0x5384af4 to 0x539013c, (0xb648 bytes) 2024-09-14T22:54:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:5120 writesame 0x2fed698 thru 0x2ffcf24 data_size 0x1(0xf88d bytes) 2024-09-14T22:54:31.303 INFO:teuthology.orchestra.run.smithi076.stdout:5121 writesame 0x7499aa2 thru 0x749c6eb data_size 0x1(0x2c4a bytes) 2024-09-14T22:54:31.311 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.235220890999997, 2024-09-14T22:54:31.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:31.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:31.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:54:31.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:54:31.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:54:31.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:54:31.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.235002995000002, 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:54:31.325 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:54:31.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:54:31.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:54:31.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:54:31.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.232319348999994, 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:54:31.331 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.332 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.332 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.332 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:54:31.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.332 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.232160934000007, 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.337 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:54:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:54:31.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:54:31.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.232133668000003, 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:54:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.231926526999999, 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:54:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.231892263000006, 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:54:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.230260216999994, 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:54:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:54:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.230235605000004, 2024-09-14T22:54:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:54:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.230101318999999, 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:54:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:54:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:54:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35965 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2054144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.229051611000003, 2024-09-14T22:54:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048887012000000001, 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35965 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087911+0000", 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087912+0000", 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0299999999999999e-07 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087918+0000", 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6710000000000002e-06 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087925+0000", 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9070000000000001e-06 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.087930+0000", 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9459999999999997e-06 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136677+0000", 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048746863000000001 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136735+0000", 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8573000000000003e-05 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136798+0000", 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2549e-05 2024-09-14T22:54:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.228949426, 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:54:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.228925521999997, 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:54:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:54:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.228892001999995, 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:54:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.228737355999996, 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:54:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:54:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:54:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.228696181999993, 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:54:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:54:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.228668021999994, 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:54:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.228478631000002, 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:54:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:54:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.312388803999994, 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:54:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.312368973000005, 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:54:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:54:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.419 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:31.421 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:31.421 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:31.419+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:31.422 INFO:teuthology.orchestra.run.smithi076.stdout:5124 trunc from 0xae63ac3 to 0x333ff2d 2024-09-14T22:54:31.422 INFO:teuthology.orchestra.run.smithi076.stdout:5125 trunc from 0x333ff2d to 0x537f8cc 2024-09-14T22:54:31.422 INFO:teuthology.orchestra.run.smithi076.stdout:5126 read 0x3f69c8c thru 0x3f6bbca (0x1f3f bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5127 write 0x8093e10 thru 0x80a2840 (0xea31 bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5128 trunc from 0x80a2841 to 0xc4c11f1 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5129 write 0xa2504db thru 0xa258187 (0x7cad bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5130 writesame 0x8128f39 thru 0x813662d data_size 0x53(0xd6f5 bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5131 read 0x3ebe199 thru 0x3ec78c8 (0x9730 bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5132 writesame 0x6653247 thru 0x66565fd data_size 0x3(0x33b7 bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5133 write 0xd1e36b7 thru 0xd1e9b6e (0x64b8 bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5134 writesame 0x867668c thru 0x867fbf7 data_size 0x49(0x956c bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5135 writesame 0xed24a7d thru 0xed3249e data_size 0x7b(0xda22 bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5137 read 0x1a7bfdb thru 0x1a7fd72 (0x3d98 bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5139 read 0xb9071 thru 0xc1877 (0x8807 bytes) 2024-09-14T22:54:31.423 INFO:teuthology.orchestra.run.smithi076.stdout:5140 read 0x7fd41fe thru 0x7fd7805 (0x3608 bytes) 2024-09-14T22:54:31.424 INFO:teuthology.orchestra.run.smithi076.stdout:5141 trunc from 0xed3249f to 0x2775df2 2024-09-14T22:54:31.427 INFO:teuthology.orchestra.run.smithi076.stdout:5143 punch from 0x22591d3 to 0x2268d48, (0xfb75 bytes) 2024-09-14T22:54:31.427 INFO:teuthology.orchestra.run.smithi076.stdout:5144 read 0xb387dc thru 0xb3e19b (0x59c0 bytes) 2024-09-14T22:54:31.427 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:31.443 INFO:teuthology.orchestra.run.smithi076.stdout:5148 writesame 0xb34bfdb thru 0xb359b9a data_size 0x40(0xdbc0 bytes) 2024-09-14T22:54:31.443 INFO:teuthology.orchestra.run.smithi076.stdout:5150 write 0x35c945f thru 0x35d8d17 (0xf8b9 bytes) 2024-09-14T22:54:31.446 INFO:teuthology.orchestra.run.smithi076.stdout:5151 clone 39 order 24 su 2097152 sc 5 2024-09-14T22:54:31.521 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:31.520+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:31.564 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:31.564 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:31.572 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:31.622 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:31.621+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:31.716 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:31.716 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:31.716 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:31.723 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:31.721+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:31.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:31.824 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:31.823+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:31.871 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:54:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.360807715999997, 2024-09-14T22:54:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:54:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:54:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:54:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.496149860999999, 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.496043925000002, 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:54:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495969224, 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495883743, 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:54:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:54:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495737357000003, 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:54:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495687357000001, 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:54:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495524193000001, 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:54:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495485883000001, 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.914 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:54:31.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:54:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:54:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495352214, 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.919 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:54:31.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.924 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495271647000003, 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.925 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:31.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495174710000001, 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.929 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:54:31.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:54:31.933 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.495123155000002, 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.934 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:31.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:31.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:54:31.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:54:31.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:54:31.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:54:31.940 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.940 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.940 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.491998526000003, 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.945 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.491866913000003, 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.946 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:54:31.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:31.950 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.491580501000001, 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.951 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:54:31.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.952 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:31.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:31.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:54:31.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:54:31.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:54:31.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:54:31.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:54:31.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:54:31.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:54:31.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:54:31.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:54:31.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:54:31.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:54:31.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:54:31.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:54:31.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:54:31.959 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.959 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.959 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.960 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:31.960 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:31.960 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.433544247, 2024-09-14T22:54:31.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:54:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:54:31.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.962 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:31.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:31.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:54:31.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:54:31.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:54:31.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:54:31.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:54:31.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:54:31.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:54:31.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:54:31.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:54:31.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:54:31.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:54:31.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:54:31.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:31.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:54:31.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:54:31.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:31.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:54:31.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:54:31.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:31.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:54:31.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:54:31.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:31.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:54:31.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:54:31.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:54:31.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:54:31.971 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.971 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.971 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.972 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:31.972 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:31.972 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.433504418999998, 2024-09-14T22:54:31.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:54:31.972 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.973 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.973 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:31.973 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.973 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.973 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.973 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:54:31.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.974 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:31.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:54:31.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:54:31.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.433488403999998, 2024-09-14T22:54:31.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:31.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:54:31.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:54:31.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:54:31.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:54:31.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:54:31.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:54:31.987 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.384360049000001, 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:31.988 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:31.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:54:31.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:31.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:31.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:54:31.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:54:31.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.994 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:31.994 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:31.994 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:31.996 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:31.995+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:31.997 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:32.097 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:32.096+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:32.118 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:32.118 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:32.118 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:32.118 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:32.128 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:32.198 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:32.197+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:32.251 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:32.251 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:32.251 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:32.251 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:32.251 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:32.262 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:32.264 INFO:teuthology.orchestra.run.smithi076.stdout:flattening image image_client.0-clone38 2024-09-14T22:54:32.306 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:32.305+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:32.375 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:32.375 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:32.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:32.375 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:32.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.375 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.375 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:32.375 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.523014083000007, 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:54:32.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.522076493, 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:54:32.380 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:54:32.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:32.385 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.521985130000004, 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:32.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:54:32.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:54:32.390 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.390 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.390 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.521340245000005, 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.392 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:32.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:54:32.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.520852865999998, 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:54:32.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:32.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:54:32.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:54:32.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.519965591000002, 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.402 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:32.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:54:32.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.406 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.519373166999998, 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.407 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:54:32.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.518640351000002, 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.412 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:54:32.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.518541201999994, 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.417 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:54:32.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.518222743999999, 2024-09-14T22:54:32.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:54:32.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.426 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.51771325, 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:54:32.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.517655391000005, 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.431 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:54:32.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.516862767999996, 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.436 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:54:32.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.516187329000005, 2024-09-14T22:54:32.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:54:32.442 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.442 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.442 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.442 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:54:32.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.446 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.516054018999995, 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:54:32.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.515437095999999, 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.451 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:54:32.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.515370239000006, 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.456 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:32.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:54:32.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.460 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.514919427999999, 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.461 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:54:32.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:32.465 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.514799194999995, 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:54:32.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:54:32.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:54:32.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:54:32.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:54:32.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.470 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 68.513481068000004, 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:54:32.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.476 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:32.478 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:32.478 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-14T22:54:32.478 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:32.476+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:32.578 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:32.577+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:32.605 INFO:teuthology.orchestra.run.smithi076.stdout:5153 read 0xc35699 thru 0xc40ea3 (0xb80b bytes) 2024-09-14T22:54:32.606 INFO:teuthology.orchestra.run.smithi076.stdout:5154 read 0x417a07d thru 0x4186932 (0xc8b6 bytes) 2024-09-14T22:54:32.606 INFO:teuthology.orchestra.run.smithi076.stdout:5156 writesame 0x5ab19f0 thru 0x5ab8b0a data_size 0x1(0x711b bytes) 2024-09-14T22:54:32.631 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:32.631 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:32.631 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:32.631 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:32.637 INFO:teuthology.orchestra.run.smithi076.stdout:5158 trunc from 0xb359b9b to 0x454c6c4 2024-09-14T22:54:32.644 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:32.669 INFO:teuthology.orchestra.run.smithi076.stdout:5159 write 0x37dbb2a thru 0x37eae9e (0xf375 bytes) 2024-09-14T22:54:32.681 INFO:teuthology.orchestra.run.smithi076.stdout:5161 writesame 0x8f61dbd thru 0x8f62eb7 data_size 0x1b(0x10fb bytes) 2024-09-14T22:54:32.681 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:32.682 INFO:teuthology.orchestra.run.smithi076.stdout:5163 write 0x885560 thru 0x8950f3 (0xfb94 bytes) 2024-09-14T22:54:32.682 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:32.680+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:32.683 INFO:teuthology.orchestra.run.smithi076.stdout:5164 read 0x2f30872 thru 0x2f32370 (0x1aff bytes) 2024-09-14T22:54:32.683 INFO:teuthology.orchestra.run.smithi076.stdout:5165 write 0x1ab18f7 thru 0x1ab94a2 (0x7bac bytes) 2024-09-14T22:54:32.685 INFO:teuthology.orchestra.run.smithi076.stdout:5166 punch from 0x8038474 to 0x80454f1, (0xd07d bytes) 2024-09-14T22:54:32.685 INFO:teuthology.orchestra.run.smithi076.stdout:5167 read 0x3af2a34 thru 0x3afb307 (0x88d4 bytes) 2024-09-14T22:54:32.693 INFO:teuthology.orchestra.run.smithi076.stdout:5168 write 0xa662e42 thru 0xa667ffa (0x51b9 bytes) 2024-09-14T22:54:32.696 INFO:teuthology.orchestra.run.smithi076.stdout:5169 punch from 0x298fae9 to 0x2996d21, (0x7238 bytes) 2024-09-14T22:54:32.696 INFO:teuthology.orchestra.run.smithi076.stdout:5172 writesame 0x83afa3 thru 0x84689e data_size 0x2(0xb8fc bytes) 2024-09-14T22:54:32.710 INFO:teuthology.orchestra.run.smithi076.stdout:5174 clone 40 order 18 su 32768 sc 11 2024-09-14T22:54:32.764 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:54:32.777 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:32.777 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:32.777 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:32.777 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:32.777 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:32.780 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:32.779+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:32.786 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:32.881 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:32.879+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.948330654999999, 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.948 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:54:32.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:54:32.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:54:32.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:32.953 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.948311617000002, 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:32.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.948275660999997, 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.958 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:54:32.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:54:32.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.948254540999997, 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.963 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:32.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.967 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.947047627000003, 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.968 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:54:32.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.945218179999998, 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.973 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:54:32.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.944663892999998, 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.978 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:54:32.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.982 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.942679175000002, 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:54:32.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:54:32.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.987 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.942326854999997, 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:54:32.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:54:32.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:54:32.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.992 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.942043411999997, 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:54:32.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:54:32.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.941755315000002, 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:32.997 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:54:32.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:54:33.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:54:33.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.941380487000004, 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:33.002 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.006 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.941059672000002, 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.007 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.927374436999997, 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:54:33.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.016 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.927039628000003, 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.017 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:54:33.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.926709465999998, 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:33.022 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:54:33.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:33.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:54:33.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:54:33.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:54:33.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:54:33.026 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.926488679999999, 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.027 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:54:33.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:33.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:33.031 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.926219379000003, 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:33.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.926032282999998, 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.036 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:54:33.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:54:33.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 55.925714124000002, 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:54:33.040 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:33.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:54:33.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:54:33.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.044 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:33.045 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:33.045 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:33.046 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:33.044+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:33.046 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:33.054 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:33.065 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:33.066 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:54:33.078 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-14T22:54:32.799199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1599741539,"num_objects":2529,"num_object_clones":484,"num_object_copies":5058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2529,"num_whiteouts":1557,"num_read":229528,"num_read_kb":1012703,"num_write":13891,"num_write_kb":542036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":77603230,"num_keys_recovered":54,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12706,"ondisk_log_size":12706,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1006120,"kb_used_data":682912,"kb_used_omap":12,"kb_used_meta":322867,"kb_avail":753968600,"statfs":{"total":773094113280,"available":772063846400,"internally_reserved":0,"allocated":699301888,"data_stored":811056203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":330616369},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":147295882,"num_objects":216,"num_object_clones":78,"num_object_copies":432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":216,"num_whiteouts":133,"num_read":15343,"num_read_kb":64876,"num_write":2163,"num_write_kb":35092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":43352392,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.629349"},"pg_stats":[{"pgid":"3.1f","version":"96'534","reported_seq":7665,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.302470+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:54:29.302470+0000","last_peered":"2024-09-14T22:54:29.302470+0000","last_clean":"2024-09-14T22:54:29.302470+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:54:29.302470+0000","last_undegraded":"2024-09-14T22:54:29.302470+0000","last_fullsized":"2024-09-14T22:54:29.302470+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:26:21.625600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":45640341,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":49,"num_read":8829,"num_read_kb":30086,"num_write":620,"num_write_kb":7298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"94'340","reported_seq":7486,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.173255+0000","last_change":"2024-09-14T22:54:28.173255+0000","last_active":"2024-09-14T22:54:28.173255+0000","last_peered":"2024-09-14T22:54:28.173255+0000","last_clean":"2024-09-14T22:54:28.173255+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:54:28.173255+0000","last_undegraded":"2024-09-14T22:54:28.173255+0000","last_fullsized":"2024-09-14T22:54:28.173255+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:36.709094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098261599999999996,"stat_sum":{"num_bytes":31336211,"num_objects":84,"num_object_clones":21,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":48,"num_read":7333,"num_read_kb":18866,"num_write":341,"num_write_kb":12322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"96'303","reported_seq":7833,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.700360+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:54:28.700360+0000","last_peered":"2024-09-14T22:54:28.700360+0000","last_clean":"2024-09-14T22:54:28.700360+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:54:28.700360+0000","last_undegraded":"2024-09-14T22:54:28.700360+0000","last_fullsized":"2024-09-14T22:54:28.700360+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:06:32.916398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":55977828,"num_objects":81,"num_object_clones":17,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":47,"num_read":7963,"num_read_kb":35359,"num_write":291,"num_write_kb":33068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"96'459","reported_seq":2454,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.541967+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:54:29.541967+0000","last_peered":"2024-09-14T22:54:29.541967+0000","last_clean":"2024-09-14T22:54:29.541967+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:54:29.541967+0000","last_undegraded":"2024-09-14T22:54:29.541967+0000","last_fullsized":"2024-09-14T22:54:29.541967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:30:23.805939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":53039446,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":38,"num_read":3568,"num_read_kb":9643,"num_write":524,"num_write_kb":8559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"94'228","reported_seq":4307,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.169260+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:54:28.169260+0000","last_peered":"2024-09-14T22:54:28.169260+0000","last_clean":"2024-09-14T22:54:28.169260+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:54:28.169260+0000","last_undegraded":"2024-09-14T22:54:28.169260+0000","last_fullsized":"2024-09-14T22:54:28.169260+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:39:51.572483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":76991017,"num_objects":69,"num_object_clones":10,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":42,"num_read":4058,"num_read_kb":51261,"num_write":232,"num_write_kb":17827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"94'339","reported_seq":11835,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.581454+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:54:29.581454+0000","last_peered":"2024-09-14T22:54:29.581454+0000","last_clean":"2024-09-14T22:54:29.581454+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:54:29.581454+0000","last_undegraded":"2024-09-14T22:54:29.581454+0000","last_fullsized":"2024-09-14T22:54:29.581454+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:06:27.925275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":59123683,"num_objects":97,"num_object_clones":26,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":54,"num_read":12302,"num_read_kb":50001,"num_write":279,"num_write_kb":28585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"94'296","reported_seq":15212,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.169788+0000","last_change":"2024-09-14T22:53:51.943155+0000","last_active":"2024-09-14T22:54:28.169788+0000","last_peered":"2024-09-14T22:54:28.169788+0000","last_clean":"2024-09-14T22:54:28.169788+0000","last_became_active":"2024-09-14T22:53:51.942663+0000","last_became_peered":"2024-09-14T22:53:51.942663+0000","last_unstale":"2024-09-14T22:54:28.169788+0000","last_undegraded":"2024-09-14T22:54:28.169788+0000","last_fullsized":"2024-09-14T22:54:28.169788+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:42.672503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":75872514,"num_objects":90,"num_object_clones":16,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":58,"num_read":15275,"num_read_kb":91712,"num_write":262,"num_write_kb":37717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"94'565","reported_seq":1636,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.171666+0000","last_change":"2024-09-14T22:54:28.171666+0000","last_active":"2024-09-14T22:54:28.171666+0000","last_peered":"2024-09-14T22:54:28.171666+0000","last_clean":"2024-09-14T22:54:28.171666+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:54:28.171666+0000","last_undegraded":"2024-09-14T22:54:28.171666+0000","last_fullsized":"2024-09-14T22:54:28.171666+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:00:49.586520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062866099999999998,"stat_sum":{"num_bytes":33861327,"num_objects":74,"num_object_clones":9,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":51,"num_read":2617,"num_read_kb":22568,"num_write":781,"num_write_kb":10013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"91'510","reported_seq":5996,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.172063+0000","last_change":"2024-09-14T22:53:38.882316+0000","last_active":"2024-09-14T22:54:28.172063+0000","last_peered":"2024-09-14T22:54:28.172063+0000","last_clean":"2024-09-14T22:54:28.172063+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:54:28.172063+0000","last_undegraded":"2024-09-14T22:54:28.172063+0000","last_fullsized":"2024-09-14T22:54:28.172063+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:34.403347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":34010799,"num_objects":62,"num_object_clones":8,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":34,"num_read":7319,"num_read_kb":33512,"num_write":716,"num_write_kb":27062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"91'297","reported_seq":8737,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.172074+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:54:28.172074+0000","last_peered":"2024-09-14T22:54:28.172074+0000","last_clean":"2024-09-14T22:54:28.172074+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:54:28.172074+0000","last_undegraded":"2024-09-14T22:54:28.172074+0000","last_fullsized":"2024-09-14T22:54:28.172074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:50:15.160158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":61201000,"num_objects":77,"num_object_clones":17,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":48,"num_read":9505,"num_read_kb":43574,"num_write":257,"num_write_kb":15493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"94'213","reported_seq":8601,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.185341+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:54:28.185341+0000","last_peered":"2024-09-14T22:54:28.185341+0000","last_clean":"2024-09-14T22:54:28.185341+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:54:28.185341+0000","last_undegraded":"2024-09-14T22:54:28.185341+0000","last_fullsized":"2024-09-14T22:54:28.185341+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:50.097468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":34741299,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":43,"num_read":9005,"num_read_kb":42017,"num_write":164,"num_write_kb":13237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"96'431","reported_seq":10412,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.603578+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:54:29.603578+0000","last_peered":"2024-09-14T22:54:29.603578+0000","last_clean":"2024-09-14T22:54:29.603578+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:54:29.603578+0000","last_undegraded":"2024-09-14T22:54:29.603578+0000","last_fullsized":"2024-09-14T22:54:29.603578+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:57:42.548136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":107682625,"num_objects":92,"num_object_clones":18,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":49,"num_read":10987,"num_read_kb":65706,"num_write":502,"num_write_kb":20654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":167,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.171402+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:54:28.171402+0000","last_peered":"2024-09-14T22:54:28.171402+0000","last_clean":"2024-09-14T22:54:28.171402+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:54:28.171402+0000","last_undegraded":"2024-09-14T22:54:28.171402+0000","last_fullsized":"2024-09-14T22:54:28.171402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"96'523","reported_seq":7154,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.586384+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:54:29.586384+0000","last_peered":"2024-09-14T22:54:29.586384+0000","last_clean":"2024-09-14T22:54:29.586384+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:54:29.586384+0000","last_undegraded":"2024-09-14T22:54:29.586384+0000","last_fullsized":"2024-09-14T22:54:29.586384+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:17:07.382895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":73946266,"num_objects":70,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":40,"num_read":8749,"num_read_kb":39042,"num_write":619,"num_write_kb":23750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":126,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.171984+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:54:28.171984+0000","last_peered":"2024-09-14T22:54:28.171984+0000","last_clean":"2024-09-14T22:54:28.171984+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:54:28.171984+0000","last_undegraded":"2024-09-14T22:54:28.171984+0000","last_fullsized":"2024-09-14T22:54:28.171984+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"96'240","reported_seq":1370,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.541366+0000","last_change":"2024-09-14T22:54:07.044085+0000","last_active":"2024-09-14T22:54:29.541366+0000","last_peered":"2024-09-14T22:54:29.541366+0000","last_clean":"2024-09-14T22:54:29.541366+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:54:29.541366+0000","last_undegraded":"2024-09-14T22:54:29.541366+0000","last_fullsized":"2024-09-14T22:54:29.541366+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:45:02.350894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":50707798,"num_objects":71,"num_object_clones":16,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":44,"num_read":961,"num_read_kb":12420,"num_write":229,"num_write_kb":16669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.185537+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:54:28.185537+0000","last_peered":"2024-09-14T22:54:28.185537+0000","last_clean":"2024-09-14T22:54:28.185537+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:54:28.185537+0000","last_undegraded":"2024-09-14T22:54:28.185537+0000","last_fullsized":"2024-09-14T22:54:28.185537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"94'530","reported_seq":12907,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.191879+0000","last_change":"2024-09-14T22:53:51.942879+0000","last_active":"2024-09-14T22:54:28.191879+0000","last_peered":"2024-09-14T22:54:28.191879+0000","last_clean":"2024-09-14T22:54:28.191879+0000","last_became_active":"2024-09-14T22:53:51.942267+0000","last_became_peered":"2024-09-14T22:53:51.942267+0000","last_unstale":"2024-09-14T22:54:28.191879+0000","last_undegraded":"2024-09-14T22:54:28.191879+0000","last_fullsized":"2024-09-14T22:54:28.191879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:08:49.720399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":33598550,"num_objects":90,"num_object_clones":16,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":55,"num_read":14239,"num_read_kb":38788,"num_write":675,"num_write_kb":19221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.168192+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:54:28.168192+0000","last_peered":"2024-09-14T22:54:28.168192+0000","last_clean":"2024-09-14T22:54:28.168192+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:54:28.168192+0000","last_undegraded":"2024-09-14T22:54:28.168192+0000","last_fullsized":"2024-09-14T22:54:28.168192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"96'316","reported_seq":14580,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.521332+0000","last_change":"2024-09-14T22:54:28.178552+0000","last_active":"2024-09-14T22:54:29.521332+0000","last_peered":"2024-09-14T22:54:29.521332+0000","last_clean":"2024-09-14T22:54:29.521332+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:54:29.521332+0000","last_undegraded":"2024-09-14T22:54:29.521332+0000","last_fullsized":"2024-09-14T22:54:29.521332+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:55:16.521985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":74854667,"num_objects":86,"num_object_clones":15,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":50,"num_read":13601,"num_read_kb":47922,"num_write":289,"num_write_kb":18867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"96'618","reported_seq":5450,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.597083+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:54:29.597083+0000","last_peered":"2024-09-14T22:54:29.597083+0000","last_clean":"2024-09-14T22:54:29.597083+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:54:29.597083+0000","last_undegraded":"2024-09-14T22:54:29.597083+0000","last_fullsized":"2024-09-14T22:54:29.597083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:22:37.321498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":53351432,"num_objects":94,"num_object_clones":20,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":58,"num_read":7829,"num_read_kb":55282,"num_write":712,"num_write_kb":30396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":169,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.171959+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:54:28.171959+0000","last_peered":"2024-09-14T22:54:28.171959+0000","last_clean":"2024-09-14T22:54:28.171959+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:54:28.171959+0000","last_undegraded":"2024-09-14T22:54:28.171959+0000","last_fullsized":"2024-09-14T22:54:28.171959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"96'254","reported_seq":1061,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.387472+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:54:29.387472+0000","last_peered":"2024-09-14T22:54:29.387472+0000","last_clean":"2024-09-14T22:54:29.387472+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:54:29.387472+0000","last_undegraded":"2024-09-14T22:54:29.387472+0000","last_fullsized":"2024-09-14T22:54:29.387472+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:34.116593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":12575758,"num_objects":59,"num_object_clones":9,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":35,"num_read":1000,"num_read_kb":8318,"num_write":258,"num_write_kb":11338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":177,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.191908+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:54:28.191908+0000","last_peered":"2024-09-14T22:54:28.191908+0000","last_clean":"2024-09-14T22:54:28.191908+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:54:28.191908+0000","last_undegraded":"2024-09-14T22:54:28.191908+0000","last_fullsized":"2024-09-14T22:54:28.191908+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"96'493","reported_seq":8479,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.396057+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:54:29.396057+0000","last_peered":"2024-09-14T22:54:29.396057+0000","last_clean":"2024-09-14T22:54:29.396057+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:54:29.396057+0000","last_undegraded":"2024-09-14T22:54:29.396057+0000","last_fullsized":"2024-09-14T22:54:29.396057+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:51:32.597165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":26518013,"num_objects":76,"num_object_clones":14,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":46,"num_read":9647,"num_read_kb":20937,"num_write":627,"num_write_kb":10667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.185149+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:54:28.185149+0000","last_peered":"2024-09-14T22:54:28.185149+0000","last_clean":"2024-09-14T22:54:28.185149+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:54:28.185149+0000","last_undegraded":"2024-09-14T22:54:28.185149+0000","last_fullsized":"2024-09-14T22:54:28.185149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'267","reported_seq":5676,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.170987+0000","last_change":"2024-09-14T22:54:18.745625+0000","last_active":"2024-09-14T22:54:28.170987+0000","last_peered":"2024-09-14T22:54:28.170987+0000","last_clean":"2024-09-14T22:54:28.170987+0000","last_became_active":"2024-09-14T22:54:18.744069+0000","last_became_peered":"2024-09-14T22:54:18.744069+0000","last_unstale":"2024-09-14T22:54:28.170987+0000","last_undegraded":"2024-09-14T22:54:28.170987+0000","last_fullsized":"2024-09-14T22:54:28.170987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:22:27.211239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":58509638,"num_objects":96,"num_object_clones":20,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":62,"num_read":5053,"num_read_kb":30758,"num_write":214,"num_write_kb":28454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":168,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.172048+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:54:28.172048+0000","last_peered":"2024-09-14T22:54:28.172048+0000","last_clean":"2024-09-14T22:54:28.172048+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:54:28.172048+0000","last_undegraded":"2024-09-14T22:54:28.172048+0000","last_fullsized":"2024-09-14T22:54:28.172048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":318,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.185244+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:54:28.185244+0000","last_peered":"2024-09-14T22:54:28.185244+0000","last_clean":"2024-09-14T22:54:28.185244+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:54:28.185244+0000","last_undegraded":"2024-09-14T22:54:28.185244+0000","last_fullsized":"2024-09-14T22:54:28.185244+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"94'433","reported_seq":8265,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.170196+0000","last_change":"2024-09-14T22:53:48.924907+0000","last_active":"2024-09-14T22:54:28.170196+0000","last_peered":"2024-09-14T22:54:28.170196+0000","last_clean":"2024-09-14T22:54:28.170196+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:54:28.170196+0000","last_undegraded":"2024-09-14T22:54:28.170196+0000","last_fullsized":"2024-09-14T22:54:28.170196+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:03:29.838215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":49431084,"num_objects":95,"num_object_clones":21,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":57,"num_read":10488,"num_read_kb":32653,"num_write":391,"num_write_kb":12535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"96'364","reported_seq":1455,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.541280+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:54:29.541280+0000","last_peered":"2024-09-14T22:54:29.541280+0000","last_clean":"2024-09-14T22:54:29.541280+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:54:29.541280+0000","last_undegraded":"2024-09-14T22:54:29.541280+0000","last_fullsized":"2024-09-14T22:54:29.541280+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:45:15.048406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":36078734,"num_objects":85,"num_object_clones":15,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":57,"num_read":1946,"num_read_kb":7231,"num_write":329,"num_write_kb":5553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'349","reported_seq":1426,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.171385+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:54:28.171385+0000","last_peered":"2024-09-14T22:54:28.171385+0000","last_clean":"2024-09-14T22:54:28.171385+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:54:28.171385+0000","last_undegraded":"2024-09-14T22:54:28.171385+0000","last_fullsized":"2024-09-14T22:54:28.171385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:44.462443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":31221787,"num_objects":63,"num_object_clones":10,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":38,"num_read":2379,"num_read_kb":8018,"num_write":377,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"96'436","reported_seq":2840,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.604422+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:54:29.604422+0000","last_peered":"2024-09-14T22:54:29.604422+0000","last_clean":"2024-09-14T22:54:29.604422+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:54:29.604422+0000","last_undegraded":"2024-09-14T22:54:29.604422+0000","last_fullsized":"2024-09-14T22:54:29.604422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:53:07.482998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":30215193,"num_objects":79,"num_object_clones":16,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":54,"num_read":4322,"num_read_kb":18812,"num_write":413,"num_write_kb":8605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"94'438","reported_seq":4988,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.177310+0000","last_change":"2024-09-14T22:54:28.177310+0000","last_active":"2024-09-14T22:54:28.177310+0000","last_peered":"2024-09-14T22:54:28.177310+0000","last_clean":"2024-09-14T22:54:28.177310+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:54:28.177310+0000","last_undegraded":"2024-09-14T22:54:28.177310+0000","last_fullsized":"2024-09-14T22:54:28.177310+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:08:13.366781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":32113789,"num_objects":83,"num_object_clones":13,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":51,"num_read":5680,"num_read_kb":21177,"num_write":462,"num_write_kb":16740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"94'260","reported_seq":6206,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.185438+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:54:28.185438+0000","last_peered":"2024-09-14T22:54:28.185438+0000","last_clean":"2024-09-14T22:54:28.185438+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:54:28.185438+0000","last_undegraded":"2024-09-14T22:54:28.185438+0000","last_fullsized":"2024-09-14T22:54:28.185438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:13:51.281278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":26665337,"num_objects":79,"num_object_clones":13,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":54,"num_read":6103,"num_read_kb":33358,"num_write":268,"num_write_kb":5426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"96'381","reported_seq":6357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.555947+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:54:29.555947+0000","last_peered":"2024-09-14T22:54:29.555947+0000","last_clean":"2024-09-14T22:54:29.555947+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:54:29.555947+0000","last_undegraded":"2024-09-14T22:54:29.555947+0000","last_fullsized":"2024-09-14T22:54:29.555947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:40:33.245308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":70725772,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":47,"num_read":7199,"num_read_kb":39618,"num_write":367,"num_write_kb":21223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"94'234","reported_seq":5899,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.185631+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:54:28.185631+0000","last_peered":"2024-09-14T22:54:28.185631+0000","last_clean":"2024-09-14T22:54:28.185631+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:54:28.185631+0000","last_undegraded":"2024-09-14T22:54:28.185631+0000","last_fullsized":"2024-09-14T22:54:28.185631+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:37:39.489218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":28566586,"num_objects":79,"num_object_clones":17,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":48,"num_read":5656,"num_read_kb":21255,"num_write":207,"num_write_kb":11550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"94'342","reported_seq":8902,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.171484+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:54:28.171484+0000","last_peered":"2024-09-14T22:54:28.171484+0000","last_clean":"2024-09-14T22:54:28.171484+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:54:28.171484+0000","last_undegraded":"2024-09-14T22:54:28.171484+0000","last_fullsized":"2024-09-14T22:54:28.171484+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:47:33.886663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":68945901,"num_objects":80,"num_object_clones":14,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":48,"num_read":10360,"num_read_kb":29026,"num_write":323,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"96'478","reported_seq":4978,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.541741+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:54:29.541741+0000","last_peered":"2024-09-14T22:54:29.541741+0000","last_clean":"2024-09-14T22:54:29.541741+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:54:29.541741+0000","last_undegraded":"2024-09-14T22:54:29.541741+0000","last_fullsized":"2024-09-14T22:54:29.541741+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:27:31.691503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":68007332,"num_objects":72,"num_object_clones":15,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":41,"num_read":5869,"num_read_kb":32977,"num_write":618,"num_write_kb":23922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"94'313","reported_seq":1126,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:28.185193+0000","last_change":"2024-09-14T22:53:38.883182+0000","last_active":"2024-09-14T22:54:28.185193+0000","last_peered":"2024-09-14T22:54:28.185193+0000","last_clean":"2024-09-14T22:54:28.185193+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:54:28.185193+0000","last_undegraded":"2024-09-14T22:54:28.185193+0000","last_fullsized":"2024-09-14T22:54:28.185193+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.546351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":47449396,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":52,"num_read":1211,"num_read_kb":10505,"num_write":360,"num_write_kb":9079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"96'802","reported_seq":5959,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-14T22:54:29.321850+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:54:29.321850+0000","last_peered":"2024-09-14T22:54:29.321850+0000","last_clean":"2024-09-14T22:54:29.321850+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:54:29.321850+0000","last_undegraded":"2024-09-14T22:54:29.321850+0000","last_fullsized":"2024-09-14T22:54:29.321850+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:11:57.358507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":56190029,"num_objects":87,"num_object_clones":16,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":59,"num_read":8359,"num_read_kb":10201,"num_write":1023,"num_write_kb":4280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1599151152,"num_objects":2525,"num_object_clones":484,"num_object_copies":5050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2525,"num_whiteouts":1557,"num_read":229412,"num_read_kb":1012603,"num_write":13730,"num_write_kb":539618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":77603230,"num_keys_recovered":54,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":689053696,"data_stored":805679194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12586,"ondisk_log_size":12586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135876,"kb_used_data":92212,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94235964,"statfs":{"total":96636764160,"available":96497627136,"internally_reserved":0,"allocated":94425088,"data_stored":108880873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80024,"kb_used_data":44676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94291816,"statfs":{"total":96636764160,"available":96554819584,"internally_reserved":0,"allocated":45748224,"data_stored":53806344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121904,"kb_used_data":82224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94249936,"statfs":{"total":96636764160,"available":96511934464,"internally_reserved":0,"allocated":84197376,"data_stored":93035143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208684,"kb_used_data":159452,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94163156,"statfs":{"total":96636764160,"available":96423071744,"internally_reserved":0,"allocated":163278848,"data_stored":185108475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542168,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130168,"kb_used_data":90656,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94241672,"statfs":{"total":96636764160,"available":96503472128,"internally_reserved":0,"allocated":92831744,"data_stored":102114762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542168,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56688,"kb_used_data":21340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315152,"statfs":{"total":96636764160,"available":96578715648,"internally_reserved":0,"allocated":21852160,"data_stored":31669025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":115704,"kb_used_data":80352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94256136,"statfs":{"total":96636764160,"available":96518283264,"internally_reserved":0,"allocated":82280448,"data_stored":105491885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542169,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":157072,"kb_used_data":112000,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94214768,"statfs":{"total":96636764160,"available":96475922432,"internally_reserved":0,"allocated":114688000,"data_stored":130949696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":112959488,"data_stored":129832161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81141760,"data_stored":104964698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20717568,"data_stored":31141857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":91697152,"data_stored":101587594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":162066432,"data_stored":184516248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":83058688,"data_stored":92507956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":44613632,"data_stored":53279176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":92798976,"data_stored":107849504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:54:33.080 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T22:54:33.080 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:54:33.080 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-14T22:54:33.080 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T22:54:33.080 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T22:54:33.080 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-14T22:54:33.146 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:33.145+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:33.152 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:33.153 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:33.153 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:33.153 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:33.162 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:33.186 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:33.187 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:33.187 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:33.187 INFO:teuthology.orchestra.run.smithi114.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T22:54:33.188 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:33.203 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:33.204 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:33.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:54:33.224 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:33.224 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:33.244 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:33.244 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:33.247 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:33.245+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:33.262 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:33.263 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:33.277 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone39 from 0xa667ffb (overlap 0x454c6c4) to 0x0 2024-09-14T22:54:33.285 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:33.285 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:33.305 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:33.305 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:33.309 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:33.309 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:33.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:33.324 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:33.325 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:33.332 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-14T22:54:33.377 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:33.376+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:33.449 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:33.447+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:33.486 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:33.486 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:33.496 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:33.550 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:33.549+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:33.556 INFO:teuthology.orchestra.run.smithi076.stdout:5175 punch from 0x6631ed1 to 0x66358a8, (0x39d7 bytes) 2024-09-14T22:54:33.559 INFO:teuthology.orchestra.run.smithi076.stdout:5177 read 0xdcbe58 thru 0xdd118e (0x5337 bytes) 2024-09-14T22:54:33.561 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:33.561 INFO:teuthology.orchestra.run.smithi076.stdout:5179 punch from 0x7d2f356 to 0x7d3a179, (0xae23 bytes) 2024-09-14T22:54:33.561 INFO:teuthology.orchestra.run.smithi076.stdout:5180 writesame 0x579a1e6 thru 0x579d0a5 data_size 0x44(0x2ec0 bytes) 2024-09-14T22:54:33.564 INFO:teuthology.orchestra.run.smithi076.stdout:5181 read 0x89cbc9d thru 0x89d1ec2 (0x6226 bytes) 2024-09-14T22:54:33.565 INFO:teuthology.orchestra.run.smithi076.stdout:5183 writesame 0x5c31a5c thru 0x5c3814c data_size 0x13(0x66f1 bytes) 2024-09-14T22:54:33.572 INFO:teuthology.orchestra.run.smithi076.stdout:5184 punch from 0x5dfa69b to 0x5dfb28e, (0xbf3 bytes) 2024-09-14T22:54:33.572 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:33.573 INFO:teuthology.orchestra.run.smithi076.stdout:5189 writesame 0x823c9a thru 0x82ad25 data_size 0x31(0x708c bytes) 2024-09-14T22:54:33.578 INFO:teuthology.orchestra.run.smithi076.stdout:5190 write 0x6291d9 thru 0x6384f6 (0xf31e bytes) 2024-09-14T22:54:33.600 INFO:teuthology.orchestra.run.smithi076.stdout:5192 trunc from 0xa667ffb to 0x7884e0 2024-09-14T22:54:33.636 INFO:teuthology.orchestra.run.smithi076.stdout:5193 write 0x36c02df thru 0x36cf4ae (0xf1d0 bytes) 2024-09-14T22:54:33.643 INFO:teuthology.orchestra.run.smithi076.stdout:5194 punch from 0x6baa73 to 0x6c2fd9, (0x8566 bytes) 2024-09-14T22:54:33.643 INFO:teuthology.orchestra.run.smithi076.stdout:5195 punch from 0x1fdda94 to 0x1fe306d, (0x55d9 bytes) 2024-09-14T22:54:33.643 INFO:teuthology.orchestra.run.smithi076.stdout:5196 writesame 0x2ba8eea thru 0x2bb5f7c data_size 0x3b(0xd093 bytes) 2024-09-14T22:54:33.648 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:33.648 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:33.650 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:33.649+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:33.659 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:33.659 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:33.659 INFO:teuthology.orchestra.run.smithi076.stdout:5199 write 0x7935d66 thru 0x7938f4c (0x31e7 bytes) 2024-09-14T22:54:33.669 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:33.677 INFO:teuthology.orchestra.run.smithi076.stdout:5200 writesame 0xce60a14 thru 0xce65895 data_size 0xd(0x4e82 bytes) 2024-09-14T22:54:33.684 INFO:teuthology.orchestra.run.smithi076.stdout:5201 write 0xeb36d10 thru 0xeb37f57 (0x1248 bytes) 2024-09-14T22:54:33.687 INFO:teuthology.orchestra.run.smithi076.stdout:5202 write 0x7c982d7 thru 0x7c9f8cb (0x75f5 bytes) 2024-09-14T22:54:33.689 INFO:teuthology.orchestra.run.smithi076.stdout:5203 punch from 0x3cafea7 to 0x3cb71ec, (0x7345 bytes) 2024-09-14T22:54:33.690 INFO:teuthology.orchestra.run.smithi076.stdout:5204 write 0x3f22e49 thru 0x3f30893 (0xda4b bytes) 2024-09-14T22:54:33.694 INFO:teuthology.orchestra.run.smithi076.stdout:5205 write 0xad9eba8 thru 0xada3bc9 (0x5022 bytes) 2024-09-14T22:54:33.697 INFO:teuthology.orchestra.run.smithi076.stdout:5206 read 0xb57d1a thru 0xb5a2ce (0x25b5 bytes) 2024-09-14T22:54:33.698 INFO:teuthology.orchestra.run.smithi076.stdout:5207 writesame 0x332da19 thru 0x332e697 data_size 0x1(0xc7f bytes) 2024-09-14T22:54:33.702 INFO:teuthology.orchestra.run.smithi076.stdout:5209 write 0x7497c82 thru 0x74a4817 (0xcb96 bytes) 2024-09-14T22:54:33.706 INFO:teuthology.orchestra.run.smithi076.stdout:5210 write 0x7898c9c thru 0x789eaeb (0x5e50 bytes) 2024-09-14T22:54:33.709 INFO:teuthology.orchestra.run.smithi076.stdout:5211 writesame 0x528a6b thru 0x53463d data_size 0x1(0xbbd3 bytes) 2024-09-14T22:54:33.740 INFO:teuthology.orchestra.run.smithi076.stdout:5212 writesame 0x8fdd678 thru 0x8fe70e8 data_size 0x9(0x9a71 bytes) 2024-09-14T22:54:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:5213 read 0xd8fa99b thru 0xd904856 (0x9ebc bytes) 2024-09-14T22:54:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:5215 read 0xaf11915 thru 0xaf18e84 (0x7570 bytes) 2024-09-14T22:54:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:5216 write 0x1950e49 thru 0x19561df (0x5397 bytes) 2024-09-14T22:54:33.750 INFO:teuthology.orchestra.run.smithi076.stdout:5218 read 0xd6c6f7d thru 0xd6d029b (0x931f bytes) 2024-09-14T22:54:33.750 INFO:teuthology.orchestra.run.smithi076.stdout:5219 punch from 0xa706b00 to 0xa710a19, (0x9f19 bytes) 2024-09-14T22:54:33.750 INFO:teuthology.orchestra.run.smithi076.stdout:5220 read 0x856d25c thru 0x8572146 (0x4eeb bytes) 2024-09-14T22:54:33.750 INFO:teuthology.orchestra.run.smithi076.stdout:5221 punch from 0xb101b39 to 0xb1076a4, (0x5b6b bytes) 2024-09-14T22:54:33.750 INFO:teuthology.orchestra.run.smithi076.stdout:5222 write 0x86901e9 thru 0x86939b5 (0x37cd bytes) 2024-09-14T22:54:33.751 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:33.750+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:33.753 INFO:teuthology.orchestra.run.smithi076.stdout:5223 read 0x90a2e1e thru 0x90af590 (0xc773 bytes) 2024-09-14T22:54:33.753 INFO:teuthology.orchestra.run.smithi076.stdout:5224 punch from 0xa645d9c to 0xa654277, (0xe4db bytes) 2024-09-14T22:54:33.753 INFO:teuthology.orchestra.run.smithi076.stdout:5225 writesame 0xa14333d thru 0xa144195 data_size 0x1(0xe59 bytes) 2024-09-14T22:54:33.758 INFO:teuthology.orchestra.run.smithi076.stdout:5227 write 0x8121441 thru 0x813025d (0xee1d bytes) 2024-09-14T22:54:33.764 INFO:teuthology.orchestra.run.smithi076.stdout:5228 writesame 0x68c50d6 thru 0x68cdb90 data_size 0x1(0x8abb bytes) 2024-09-14T22:54:33.784 INFO:teuthology.orchestra.run.smithi076.stdout:5229 write 0x29e9e1c thru 0x29f8d75 (0xef5a bytes) 2024-09-14T22:54:33.788 INFO:teuthology.orchestra.run.smithi076.stdout:5230 punch from 0x935ec96 to 0x936c1b4, (0xd51e bytes) 2024-09-14T22:54:33.788 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:33.791 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:33.791 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:33.804 INFO:teuthology.orchestra.run.smithi076.stdout:5233 trunc from 0xeb37f58 to 0x13e887e 2024-09-14T22:54:33.805 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:33.824 INFO:teuthology.orchestra.run.smithi076.stdout:5234 trunc from 0x13e887e to 0x3b7d026 2024-09-14T22:54:33.844 INFO:teuthology.orchestra.run.smithi076.stdout:5235 write 0xc6e6c90 thru 0xc6e77e3 (0xb54 bytes) 2024-09-14T22:54:33.847 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:33.847 INFO:teuthology.orchestra.run.smithi076.stdout:5239 read 0x2216457 thru 0x22249f5 (0xe59f bytes) 2024-09-14T22:54:33.852 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:33.851+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:33.855 INFO:teuthology.orchestra.run.smithi076.stdout:5240 write 0xee06950 thru 0xee078d4 (0xf85 bytes) 2024-09-14T22:54:33.858 INFO:teuthology.orchestra.run.smithi076.stdout:5241 read 0xcbb9419 thru 0xcbc53f3 (0xbfdb bytes) 2024-09-14T22:54:33.858 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:33.858 INFO:teuthology.orchestra.run.smithi076.stdout:5243 writesame 0x1a2bece thru 0x1a37ba0 data_size 0x29(0xbcd3 bytes) 2024-09-14T22:54:33.863 INFO:teuthology.orchestra.run.smithi076.stdout:5244 write 0xe071eaf thru 0xe076fd0 (0x5122 bytes) 2024-09-14T22:54:33.869 INFO:teuthology.orchestra.run.smithi076.stdout:5247 write 0x16f44bc thru 0x16fd0e7 (0x8c2c bytes) 2024-09-14T22:54:33.878 INFO:teuthology.orchestra.run.smithi076.stdout:5248 write 0x19b8d9d thru 0x19bcda8 (0x400c bytes) 2024-09-14T22:54:33.895 INFO:teuthology.orchestra.run.smithi076.stdout:5250 trunc from 0xee078d5 to 0x3a7473c 2024-09-14T22:54:33.916 INFO:teuthology.orchestra.run.smithi076.stdout:5251 trunc from 0x3a7473c to 0xa97cea7 2024-09-14T22:54:33.921 INFO:teuthology.orchestra.run.smithi076.stdout:5252 read 0x156bd1e thru 0x156c6d7 (0x9ba bytes) 2024-09-14T22:54:33.922 INFO:teuthology.orchestra.run.smithi076.stdout:5253 read 0x64fc994 thru 0x6504a1b (0x8088 bytes) 2024-09-14T22:54:33.922 INFO:teuthology.orchestra.run.smithi076.stdout:5254 writesame 0x65f3024 thru 0x65f8119 data_size 0x2b(0x50f6 bytes) 2024-09-14T22:54:33.927 INFO:teuthology.orchestra.run.smithi076.stdout:5255 writesame 0x5b4164d thru 0x5b4fdc7 data_size 0x1(0xe77b bytes) 2024-09-14T22:54:33.939 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:33.939 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:33.949 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:33.964 INFO:teuthology.orchestra.run.smithi076.stdout:5256 write 0x6ede58b thru 0x6ee573d (0x71b3 bytes) 2024-09-14T22:54:33.965 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:33.963+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:33.970 INFO:teuthology.orchestra.run.smithi076.stdout:5257 write 0x409f17a thru 0x409f2d9 (0x160 bytes) 2024-09-14T22:54:33.973 INFO:teuthology.orchestra.run.smithi076.stdout:5258 read 0x1bea6cb thru 0x1bed929 (0x325f bytes) 2024-09-14T22:54:33.974 INFO:teuthology.orchestra.run.smithi076.stdout:5260 trunc from 0xa97cea7 to 0xae6ba16 2024-09-14T22:54:33.979 INFO:teuthology.orchestra.run.smithi076.stdout:5261 writesame 0x703d0e5 thru 0x7041cea data_size 0x25(0x4c06 bytes) 2024-09-14T22:54:33.981 INFO:teuthology.orchestra.run.smithi076.stdout:5262 punch from 0x64e0813 to 0x64ecc78, (0xc465 bytes) 2024-09-14T22:54:33.981 INFO:teuthology.orchestra.run.smithi076.stdout:5263 writesame 0x5555136 thru 0x555ba89 data_size 0x6b(0x6954 bytes) 2024-09-14T22:54:33.986 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:33.993 INFO:teuthology.orchestra.run.smithi076.stdout:5266 writesame 0xc9de22f thru 0xc9e3c33 data_size 0xb(0x5a05 bytes) 2024-09-14T22:54:33.997 INFO:teuthology.orchestra.run.smithi076.stdout:5268 trunc from 0xc9e3c34 to 0x8ffcc02 2024-09-14T22:54:34.005 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.005 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.005 INFO:teuthology.orchestra.run.smithi076.stdout:5271 write 0x26a51d9 thru 0x26a7172 (0x1f9a bytes) 2024-09-14T22:54:34.008 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.008 INFO:teuthology.orchestra.run.smithi076.stdout:5273 read 0x3f76cde thru 0x3f84bc1 (0xdee4 bytes) 2024-09-14T22:54:34.008 INFO:teuthology.orchestra.run.smithi076.stdout:5274 write 0x4e2f9e4 thru 0x4e31ea3 (0x24c0 bytes) 2024-09-14T22:54:34.012 INFO:teuthology.orchestra.run.smithi076.stdout:5275 write 0x75a6d45 thru 0x75adcb5 (0x6f71 bytes) 2024-09-14T22:54:34.018 INFO:teuthology.orchestra.run.smithi076.stdout:5276 punch from 0x34e1fac to 0x34e93e3, (0x7437 bytes) 2024-09-14T22:54:34.022 INFO:teuthology.orchestra.run.smithi076.stdout:5278 trunc from 0x8ffcc02 to 0x58c751e 2024-09-14T22:54:34.033 INFO:teuthology.orchestra.run.smithi076.stdout:5279 read 0x83d8bd thru 0x846b5d (0x92a1 bytes) 2024-09-14T22:54:34.044 INFO:teuthology.orchestra.run.smithi076.stdout:5280 write 0xab56582 thru 0xab5cb76 (0x65f5 bytes) 2024-09-14T22:54:34.051 INFO:teuthology.orchestra.run.smithi076.stdout:5282 write 0x201cbad thru 0x202c2f3 (0xf747 bytes) 2024-09-14T22:54:34.054 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:34.053+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:34.056 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.056 INFO:teuthology.orchestra.run.smithi076.stdout:5284 write 0xd86639 thru 0xd8a280 (0x3c48 bytes) 2024-09-14T22:54:34.060 INFO:teuthology.orchestra.run.smithi076.stdout:5285 write 0x14975a4 thru 0x14a3f4f (0xc9ac bytes) 2024-09-14T22:54:34.064 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.065 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:34.065 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:34.068 INFO:teuthology.orchestra.run.smithi076.stdout:5287 trunc from 0xab5cb77 to 0xdf84b8a 2024-09-14T22:54:34.073 INFO:teuthology.orchestra.run.smithi076.stdout:5291 writesame 0x5092a0d thru 0x5092f5f data_size 0x1(0x553 bytes) 2024-09-14T22:54:34.075 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:34.078 INFO:teuthology.orchestra.run.smithi076.stdout:5293 write 0x467b8a thru 0x46cc86 (0x50fd bytes) 2024-09-14T22:54:34.083 INFO:teuthology.orchestra.run.smithi076.stdout:5294 write 0xa28b300 thru 0xa296a1c (0xb71d bytes) 2024-09-14T22:54:34.088 INFO:teuthology.orchestra.run.smithi076.stdout:5295 write 0x39ca53e thru 0x39ced3b (0x47fe bytes) 2024-09-14T22:54:34.091 INFO:teuthology.orchestra.run.smithi076.stdout:5296 punch from 0x9480508 to 0x9487c43, (0x773b bytes) 2024-09-14T22:54:34.099 INFO:teuthology.orchestra.run.smithi076.stdout:5297 trunc from 0xdf84b8a to 0x74b5760 2024-09-14T22:54:34.119 INFO:teuthology.orchestra.run.smithi076.stdout:5299 writesame 0xb738881 thru 0xb73dfd1 data_size 0x1(0x5751 bytes) 2024-09-14T22:54:34.126 INFO:teuthology.orchestra.run.smithi076.stdout:5301 write 0xd38d82e thru 0xd39d598 (0xfd6b bytes) 2024-09-14T22:54:34.131 INFO:teuthology.orchestra.run.smithi076.stdout:5302 writesame 0xbaed5d0 thru 0xbaf0384 data_size 0x1(0x2db5 bytes) 2024-09-14T22:54:34.139 INFO:teuthology.orchestra.run.smithi076.stdout:5304 punch from 0xbfa133e to 0xbfaa0d0, (0x8d92 bytes) 2024-09-14T22:54:34.139 INFO:teuthology.orchestra.run.smithi076.stdout:5305 writesame 0x5871d23 thru 0x587eac3 data_size 0x3(0xcda1 bytes) 2024-09-14T22:54:34.149 INFO:teuthology.orchestra.run.smithi076.stdout:5306 writesame 0x7f2d330 thru 0x7f35b6a data_size 0x5d(0x883b bytes) 2024-09-14T22:54:34.153 INFO:teuthology.orchestra.run.smithi076.stdout:5308 write 0x7c0f9ba thru 0x7c1e508 (0xeb4f bytes) 2024-09-14T22:54:34.154 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:34.153+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:34.158 INFO:teuthology.orchestra.run.smithi076.stdout:5310 read 0x50c89d6 thru 0x50cec47 (0x6272 bytes) 2024-09-14T22:54:34.158 INFO:teuthology.orchestra.run.smithi076.stdout:5311 read 0x9890ff7 thru 0x989cd2b (0xbd35 bytes) 2024-09-14T22:54:34.159 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.159 INFO:teuthology.orchestra.run.smithi076.stdout:5313 write 0xb221d4e thru 0xb226aef (0x4da2 bytes) 2024-09-14T22:54:34.162 INFO:teuthology.orchestra.run.smithi076.stdout:5314 write 0xb4327fb thru 0xb442237 (0xfa3d bytes) 2024-09-14T22:54:34.167 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.168 INFO:teuthology.orchestra.run.smithi076.stdout:5316 writesame 0x52620f6 thru 0x5271419 data_size 0xb6(0xf324 bytes) 2024-09-14T22:54:34.174 INFO:teuthology.orchestra.run.smithi076.stdout:5317 write 0x822ace1 thru 0x82315d1 (0x68f1 bytes) 2024-09-14T22:54:34.184 INFO:teuthology.orchestra.run.smithi076.stdout:5318 write 0xbb0d16d thru 0xbb16044 (0x8ed8 bytes) 2024-09-14T22:54:34.189 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:34.190 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:34.190 INFO:teuthology.orchestra.run.smithi076.stdout:5319 read 0x31b16d8 thru 0x31b9112 (0x7a3b bytes) 2024-09-14T22:54:34.191 INFO:teuthology.orchestra.run.smithi076.stdout:5321 punch from 0x38de448 to 0x38de590, (0x148 bytes) 2024-09-14T22:54:34.191 INFO:teuthology.orchestra.run.smithi076.stdout:5324 read 0x59a5df1 thru 0x59ad22c (0x743c bytes) 2024-09-14T22:54:34.191 INFO:teuthology.orchestra.run.smithi076.stdout:5325 read 0x85d0849 thru 0x85d3739 (0x2ef1 bytes) 2024-09-14T22:54:34.191 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.191 INFO:teuthology.orchestra.run.smithi076.stdout:5328 write 0x313d148 thru 0x3143ca9 (0x6b62 bytes) 2024-09-14T22:54:34.198 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.198 INFO:teuthology.orchestra.run.smithi076.stdout:5333 read 0xaa79fcb thru 0xaa87cd8 (0xdd0e bytes) 2024-09-14T22:54:34.199 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:34.214 INFO:teuthology.orchestra.run.smithi076.stdout:5336 trunc from 0xd39d599 to 0x165746 2024-09-14T22:54:34.215 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:54:34.256 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:34.254+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:34.285 INFO:teuthology.orchestra.run.smithi076.stdout:5337 write 0x8cbdac thru 0x8ccd1b (0xf70 bytes) 2024-09-14T22:54:34.305 INFO:teuthology.orchestra.run.smithi076.stdout:5339 write 0xa827151 thru 0xa82e2f1 (0x71a1 bytes) 2024-09-14T22:54:34.309 INFO:teuthology.orchestra.run.smithi076.stdout:5342 write 0x1989d15 thru 0x1994d6d (0xb059 bytes) 2024-09-14T22:54:34.312 INFO:teuthology.orchestra.run.smithi076.stdout:5343 read 0x679d151 thru 0x67a2309 (0x51b9 bytes) 2024-09-14T22:54:34.312 INFO:teuthology.orchestra.run.smithi076.stdout:5345 punch from 0x3988205 to 0x3993395, (0xb190 bytes) 2024-09-14T22:54:34.317 INFO:teuthology.orchestra.run.smithi076.stdout:5346 write 0xb500d95 thru 0xb506823 (0x5a8f bytes) 2024-09-14T22:54:34.320 INFO:teuthology.orchestra.run.smithi076.stdout:5347 read 0x252521d thru 0x253311d (0xdf01 bytes) 2024-09-14T22:54:34.321 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.321 INFO:teuthology.orchestra.run.smithi076.stdout:5349 writesame 0x8558c72 thru 0x855923b data_size 0x26(0x5ca bytes) 2024-09-14T22:54:34.321 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:34.322 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:34.323 INFO:teuthology.orchestra.run.smithi076.stdout:5350 read 0x65cc241 thru 0x65db50a (0xf2ca bytes) 2024-09-14T22:54:34.324 INFO:teuthology.orchestra.run.smithi076.stdout:5351 write 0x163d8fd thru 0x1645bd2 (0x82d6 bytes) 2024-09-14T22:54:34.327 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.327 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:34.332 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:34.337 INFO:teuthology.orchestra.run.smithi076.stdout:5354 trunc from 0xb506824 to 0x38a3440 2024-09-14T22:54:34.347 INFO:teuthology.orchestra.run.smithi076.stdout:5355 read 0x35e5ba2 thru 0x35f492f (0xed8e bytes) 2024-09-14T22:54:34.347 INFO:teuthology.orchestra.run.smithi076.stdout:5356 punch from 0x9dc509 to 0x9e2eb6, (0x69ad bytes) 2024-09-14T22:54:34.347 INFO:teuthology.orchestra.run.smithi076.stdout:5357 write 0x299406d thru 0x29a0f43 (0xced7 bytes) 2024-09-14T22:54:34.356 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:34.355+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:34.357 INFO:teuthology.orchestra.run.smithi076.stdout:5358 read 0x277a743 thru 0x2786932 (0xc1f0 bytes) 2024-09-14T22:54:34.357 INFO:teuthology.orchestra.run.smithi076.stdout:5359 write 0xe13908 thru 0xe1885b (0x4f54 bytes) 2024-09-14T22:54:34.376 INFO:teuthology.orchestra.run.smithi076.stdout:5360 write 0xc37ed18 thru 0xc384017 (0x5300 bytes) 2024-09-14T22:54:34.380 INFO:teuthology.orchestra.run.smithi076.stdout:5361 write 0x69da2af thru 0x69df909 (0x565b bytes) 2024-09-14T22:54:34.385 INFO:teuthology.orchestra.run.smithi076.stdout:5362 trunc from 0xc384018 to 0xde870fb 2024-09-14T22:54:34.391 INFO:teuthology.orchestra.run.smithi076.stdout:5364 read 0x54b5d59 thru 0x54baf7d (0x5225 bytes) 2024-09-14T22:54:34.391 INFO:teuthology.orchestra.run.smithi076.stdout:5365 write 0x60d2968 thru 0x60d7727 (0x4dc0 bytes) 2024-09-14T22:54:34.394 INFO:teuthology.orchestra.run.smithi076.stdout:5366 read 0x578b2a9 thru 0x578d409 (0x2161 bytes) 2024-09-14T22:54:34.394 INFO:teuthology.orchestra.run.smithi076.stdout:5367 read 0x29d44a3 thru 0x29e0435 (0xbf93 bytes) 2024-09-14T22:54:34.394 INFO:teuthology.orchestra.run.smithi076.stdout:5368 write 0x142c9d0 thru 0x1430637 (0x3c68 bytes) 2024-09-14T22:54:34.401 INFO:teuthology.orchestra.run.smithi076.stdout:5369 write 0xaaf1b78 thru 0xab014a4 (0xf92d bytes) 2024-09-14T22:54:34.408 INFO:teuthology.orchestra.run.smithi076.stdout:5371 write 0x331d927 thru 0x332c7a4 (0xee7e bytes) 2024-09-14T22:54:34.415 INFO:teuthology.orchestra.run.smithi076.stdout:5374 clone 41 order 19 su 131072 sc 12 2024-09-14T22:54:34.457 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:34.456+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:34.479 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:34.479 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:34.487 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:34.558 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:34.557+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:34.630 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:34.630 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:34.639 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:34.659 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:34.658+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:34.741 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:34.741 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:34.749 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:34.792 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:34.791+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:34.853 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:34.854 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:34.861 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:34.860+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:34.862 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:34.962 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:34.960+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:35.006 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:35.006 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:35.015 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:35.062 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:35.061+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:35.117 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:35.117 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:35.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:35.169 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:35.168+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:35.235 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:35.235 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:35.244 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:35.283 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone40 from 0xde870fb (overlap 0x165746) to 0xb2ba3 2024-09-14T22:54:35.285 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:35.284+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:35.337 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:35.364 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:35.363+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:35.370 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-14T22:54:35.372 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:35.372 INFO:teuthology.orchestra.run.smithi076.stdout:5376 read 0x1e79fcc thru 0x1e7d76d (0x37a2 bytes) 2024-09-14T22:54:35.372 INFO:teuthology.orchestra.run.smithi076.stdout:5379 writesame 0x9278457 thru 0x927b15c data_size 0x33(0x2d06 bytes) 2024-09-14T22:54:35.379 INFO:teuthology.orchestra.run.smithi076.stdout:5380 read 0x95c2f5c thru 0x95d2652 (0xf6f7 bytes) 2024-09-14T22:54:35.379 INFO:teuthology.orchestra.run.smithi076.stdout:5382 trunc from 0xde870fb to 0xd9ba078 2024-09-14T22:54:35.400 INFO:teuthology.orchestra.run.smithi076.stdout:5383 read 0x98c33bd thru 0x98c9652 (0x6296 bytes) 2024-09-14T22:54:35.400 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:35.400 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:35.405 INFO:teuthology.orchestra.run.smithi076.stdout:5385 write 0xd9afb01 thru 0xd9be8b8 (0xedb8 bytes) 2024-09-14T22:54:35.406 INFO:teuthology.orchestra.run.smithi076.stdout:5386 write 0x989be3d thru 0x98a550e (0x96d2 bytes) 2024-09-14T22:54:35.409 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:35.411 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:35.411 INFO:teuthology.orchestra.run.smithi076.stdout:5388 write 0xafe70f3 thru 0xafeb057 (0x3f65 bytes) 2024-09-14T22:54:35.418 INFO:teuthology.orchestra.run.smithi076.stdout:5389 write 0xde498ee thru 0xde4cd74 (0x3487 bytes) 2024-09-14T22:54:35.421 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:35.428 INFO:teuthology.orchestra.run.smithi076.stdout:5391 trunc from 0xde4cd75 to 0x8357a08 2024-09-14T22:54:35.446 INFO:teuthology.orchestra.run.smithi076.stdout:5392 write 0x4423501 thru 0x442fc70 (0xc770 bytes) 2024-09-14T22:54:35.462 INFO:teuthology.orchestra.run.smithi076.stdout:5395 write 0xe666bae thru 0xe66877f (0x1bd2 bytes) 2024-09-14T22:54:35.465 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:35.466 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:35.466 INFO:teuthology.orchestra.run.smithi076.stdout:5398 read 0x631f631 thru 0x6325da5 (0x6775 bytes) 2024-09-14T22:54:35.466 INFO:teuthology.orchestra.run.smithi076.stdout:5399 write 0x9660114 thru 0x966f90c (0xf7f9 bytes) 2024-09-14T22:54:35.469 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:35.469 INFO:teuthology.orchestra.run.smithi076.stdout:5401 read 0x6caaed6 thru 0x6cb72ab (0xc3d6 bytes) 2024-09-14T22:54:35.469 INFO:teuthology.orchestra.run.smithi076.stdout:5402 punch from 0x4a4a256 to 0x4a4fbba, (0x5964 bytes) 2024-09-14T22:54:35.469 INFO:teuthology.orchestra.run.smithi076.stdout:5403 punch from 0x8441850 to 0x844ff9d, (0xe74d bytes) 2024-09-14T22:54:35.469 INFO:teuthology.orchestra.run.smithi076.stdout:5404 read 0x80ab02 thru 0x81a8aa (0xfda9 bytes) 2024-09-14T22:54:35.469 INFO:teuthology.orchestra.run.smithi076.stdout:5405 read 0x6b56528 thru 0x6b59852 (0x332b bytes) 2024-09-14T22:54:35.469 INFO:teuthology.orchestra.run.smithi076.stdout:5409 write 0xc14d99d thru 0xc1513c2 (0x3a26 bytes) 2024-09-14T22:54:35.472 INFO:teuthology.orchestra.run.smithi076.stdout:5410 read 0xc7d08a7 thru 0xc7d40fd (0x3857 bytes) 2024-09-14T22:54:35.474 INFO:teuthology.orchestra.run.smithi076.stdout:5412 trunc from 0xe668780 to 0xcccd980 2024-09-14T22:54:35.476 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:35.476 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:35.486 INFO:teuthology.orchestra.run.smithi076.stdout:5413 read 0xc8a2ec5 thru 0xc8af19b (0xc2d7 bytes) 2024-09-14T22:54:35.486 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:35.486 INFO:teuthology.orchestra.run.smithi076.stdout:5415 punch from 0x3258afa to 0x325b7ce, (0x2cd4 bytes) 2024-09-14T22:54:35.487 INFO:teuthology.orchestra.run.smithi076.stdout:5416 punch from 0x4a11bf to 0x4b08b5, (0xf6f6 bytes) 2024-09-14T22:54:35.487 INFO:teuthology.orchestra.run.smithi076.stdout:5417 punch from 0x95fdd62 to 0x96050c7, (0x7365 bytes) 2024-09-14T22:54:35.487 INFO:teuthology.orchestra.run.smithi076.stdout:5418 read 0x44acc14 thru 0x44b2226 (0x5613 bytes) 2024-09-14T22:54:35.487 INFO:teuthology.orchestra.run.smithi076.stdout:5419 writesame 0xbd950c4 thru 0xbd97033 data_size 0x8(0x1f70 bytes) 2024-09-14T22:54:35.489 INFO:teuthology.orchestra.run.smithi076.stdout:5422 punch from 0x7c783c5 to 0x7c7d540, (0x517b bytes) 2024-09-14T22:54:35.489 INFO:teuthology.orchestra.run.smithi076.stdout:5423 writesame 0x2a5e6ee thru 0x2a6d7a2 data_size 0x1(0xf0b5 bytes) 2024-09-14T22:54:35.493 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:35.492+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:35.494 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:35.495 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:35.512 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:35.512 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:35.528 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:35.529 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:35.532 INFO:teuthology.orchestra.run.smithi076.stdout:5424 write 0xd185b7e thru 0xd18cdc6 (0x7249 bytes) 2024-09-14T22:54:35.537 INFO:teuthology.orchestra.run.smithi076.stdout:5426 read 0xa018461 thru 0xa01c4c5 (0x4065 bytes) 2024-09-14T22:54:35.537 INFO:teuthology.orchestra.run.smithi076.stdout:5427 write 0x896a46b thru 0x8977320 (0xceb6 bytes) 2024-09-14T22:54:35.539 INFO:teuthology.orchestra.run.smithi076.stdout:5429 write 0x551fbee thru 0x552388a (0x3c9d bytes) 2024-09-14T22:54:35.547 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:35.547 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:35.547 INFO:teuthology.orchestra.run.smithi076.stdout:5430 writesame 0x56983aa thru 0x56a635c data_size 0x65(0xdfb3 bytes) 2024-09-14T22:54:35.550 INFO:teuthology.orchestra.run.smithi076.stdout:5434 write 0xccb7300 thru 0xccc2c20 (0xb921 bytes) 2024-09-14T22:54:35.558 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:35.558 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:35.560 INFO:teuthology.orchestra.run.smithi076.stdout:5435 write 0xe7742e6 thru 0xe780f34 (0xcc4f bytes) 2024-09-14T22:54:35.565 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:35.564+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:35.567 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:35.567 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:35.568 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:35.572 INFO:teuthology.orchestra.run.smithi076.stdout:5436 write 0xe8ad0b6 thru 0xe8b2f90 (0x5edb bytes) 2024-09-14T22:54:35.576 INFO:teuthology.orchestra.run.smithi076.stdout:5437 write 0x881a6ce thru 0x881cda2 (0x26d5 bytes) 2024-09-14T22:54:35.579 INFO:teuthology.orchestra.run.smithi076.stdout:5438 read 0x5ef678 thru 0x5f55ca (0x5f53 bytes) 2024-09-14T22:54:35.579 INFO:teuthology.orchestra.run.smithi076.stdout:5439 read 0x21d7c3e thru 0x21e6be4 (0xefa7 bytes) 2024-09-14T22:54:35.579 INFO:teuthology.orchestra.run.smithi076.stdout:5440 read 0x97bbe6f thru 0x97ca323 (0xe4b5 bytes) 2024-09-14T22:54:35.579 INFO:teuthology.orchestra.run.smithi076.stdout:5441 read 0x17f8c5d thru 0x1804df7 (0xc19b bytes) 2024-09-14T22:54:35.580 INFO:teuthology.orchestra.run.smithi076.stdout:5442 read 0xd478b78 thru 0xd482ce2 (0xa16b bytes) 2024-09-14T22:54:35.580 INFO:teuthology.orchestra.run.smithi076.stdout:5444 write 0xcdb231b thru 0xcdbaf6f (0x8c55 bytes) 2024-09-14T22:54:35.584 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:35.584 INFO:teuthology.orchestra.run.smithi076.stdout:5448 write 0x289a65b thru 0x28a5cfb (0xb6a1 bytes) 2024-09-14T22:54:35.585 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:35.585 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:35.591 INFO:teuthology.orchestra.run.smithi076.stdout:5449 write 0xad559f8 thru 0xad6457c (0xeb85 bytes) 2024-09-14T22:54:35.595 INFO:teuthology.orchestra.run.smithi076.stdout:5451 flatten 2024-09-14T22:54:35.610 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:35.610 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:35.628 INFO:teuthology.orchestra.run.smithi076.stdout:5453 write 0xe181cdd thru 0xe18ddcc (0xc0f0 bytes) 2024-09-14T22:54:35.631 INFO:teuthology.orchestra.run.smithi076.stdout:5455 writesame 0x18fddc8 thru 0x190c327 data_size 0x14(0xe560 bytes) 2024-09-14T22:54:35.637 INFO:teuthology.orchestra.run.smithi076.stdout:5456 read 0x7abd35d thru 0x7ac02a8 (0x2f4c bytes) 2024-09-14T22:54:35.642 INFO:teuthology.orchestra.run.smithi076.stdout:5457 trunc from 0xe8b2f91 to 0xaf60119 2024-09-14T22:54:35.658 INFO:teuthology.orchestra.run.smithi076.stdout:5459 trunc from 0xaf60119 to 0xe500291 2024-09-14T22:54:35.663 INFO:teuthology.orchestra.run.smithi076.stdout:5460 read 0x441a27f thru 0x441f5eb (0x536d bytes) 2024-09-14T22:54:35.664 INFO:teuthology.orchestra.run.smithi076.stdout:5461 writesame 0xdfad063 thru 0xdfb06f6 data_size 0x7(0x3694 bytes) 2024-09-14T22:54:35.667 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:35.665+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:35.667 INFO:teuthology.orchestra.run.smithi076.stdout:5462 writesame 0x3513701 thru 0x351a3cc data_size 0x21(0x6ccc bytes) 2024-09-14T22:54:35.670 INFO:teuthology.orchestra.run.smithi076.stdout:5464 write 0x2d7a2be thru 0x2d81e21 (0x7b64 bytes) 2024-09-14T22:54:35.673 INFO:teuthology.orchestra.run.smithi076.stdout:5465 write 0x8cfe5d8 thru 0x8d00b78 (0x25a1 bytes) 2024-09-14T22:54:35.676 INFO:teuthology.orchestra.run.smithi076.stdout:5466 writesame 0x76f5212 thru 0x7701825 data_size 0xe(0xc614 bytes) 2024-09-14T22:54:35.681 INFO:teuthology.orchestra.run.smithi076.stdout:5469 read 0xd181cda thru 0xd18ba97 (0x9dbe bytes) 2024-09-14T22:54:35.691 INFO:teuthology.orchestra.run.smithi076.stdout:5470 trunc from 0xe500291 to 0x62bd688 2024-09-14T22:54:35.697 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:35.697 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:35.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.697 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:199584 3.1f 3:f801780f:::rbd_data.1135302df8d.0000000000000114:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-14T22:54:35.697 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:35.695393+0000", 2024-09-14T22:54:35.697 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.0010977350000000001, 2024-09-14T22:54:35.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011211109999999999, 2024-09-14T22:54:35.697 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "waiting for sub ops", 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 199584 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695393+0000", 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695393+0000", 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695393+0000", 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7599999999999996e-07 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695398+0000", 2024-09-14T22:54:35.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3030000000000001e-06 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695408+0000", 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0397000000000001e-05 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695416+0000", 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5209999999999997e-06 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:35.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695471+0000", 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5526999999999999e-05 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695534+0000", 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2581000000000003e-05 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 5", 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695596+0000", 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2525000000000002e-05 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.696513+0000", 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00091657100000000003 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:199587 3.6 3:63e39ec9:::rbd_data.1135302df8d.0000000000000156:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:35.695587+0000", 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.00090339000000000001, 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00098112700000000004, 2024-09-14T22:54:35.702 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "waiting for sub ops", 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 199587 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695587+0000", 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695587+0000", 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695588+0000", 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4499999999999996e-07 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:35.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695592+0000", 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5480000000000003e-06 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695604+0000", 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1891e-05 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695611+0000", 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6839999999999999e-06 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695684+0000", 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2658999999999997e-05 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695768+0000", 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4201000000000002e-05 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:35.695879+0000", 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111239 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 2 2024-09-14T22:54:35.707 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:35.708 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:35.711 INFO:teuthology.orchestra.run.smithi076.stdout:5471 writesame 0x81002e3 thru 0x810e374 data_size 0x5(0xe092 bytes) 2024-09-14T22:54:35.712 INFO:teuthology.orchestra.run.smithi076.stdout:5473 punch from 0xf48223 to 0xf551ef, (0xcfcc bytes) 2024-09-14T22:54:35.716 INFO:teuthology.orchestra.run.smithi076.stdout:5475 trunc from 0x810e375 to 0x3c260d1 2024-09-14T22:54:35.727 INFO:teuthology.orchestra.run.smithi076.stdout:5476 trunc from 0x3c260d1 to 0x4c5ac41 2024-09-14T22:54:35.738 INFO:teuthology.orchestra.run.smithi076.stdout:5477 trunc from 0x4c5ac41 to 0x9728429 2024-09-14T22:54:35.744 INFO:teuthology.orchestra.run.smithi076.stdout:5478 read 0x7526f8a thru 0x7533291 (0xc308 bytes) 2024-09-14T22:54:35.744 INFO:teuthology.orchestra.run.smithi076.stdout:5479 punch from 0x686508e to 0x6868b86, (0x3af8 bytes) 2024-09-14T22:54:35.746 INFO:teuthology.orchestra.run.smithi076.stdout:5480 trunc from 0x9728429 to 0xb9712c4 2024-09-14T22:54:35.753 INFO:teuthology.orchestra.run.smithi076.stdout:5481 read 0x64a4fe1 thru 0x64ac2ee (0x730e bytes) 2024-09-14T22:54:35.753 INFO:teuthology.orchestra.run.smithi076.stdout:5483 punch from 0xb01e29e to 0xb02caa3, (0xe805 bytes) 2024-09-14T22:54:35.755 INFO:teuthology.orchestra.run.smithi076.stdout:5484 trunc from 0xb9712c4 to 0xd8617c8 2024-09-14T22:54:35.760 INFO:teuthology.orchestra.run.smithi076.stdout:5485 write 0x648a0fd thru 0x6493fa8 (0x9eac bytes) 2024-09-14T22:54:35.763 INFO:teuthology.orchestra.run.smithi076.stdout:5486 writesame 0xcb08860 thru 0xcb0b30f data_size 0x8(0x2ab0 bytes) 2024-09-14T22:54:35.767 INFO:teuthology.orchestra.run.smithi076.stdout:5488 writesame 0xc72664f thru 0xc72e017 data_size 0x1(0x79c9 bytes) 2024-09-14T22:54:35.768 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:35.766+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:35.795 INFO:teuthology.orchestra.run.smithi076.stdout:5489 write 0xec0565e thru 0xec06ff7 (0x199a bytes) 2024-09-14T22:54:35.798 INFO:teuthology.orchestra.run.smithi076.stdout:5492 read 0xb559e4f thru 0xb56813f (0xe2f1 bytes) 2024-09-14T22:54:35.798 INFO:teuthology.orchestra.run.smithi076.stdout:5493 read 0x8e9b030 thru 0x8ea9cbd (0xec8e bytes) 2024-09-14T22:54:35.798 INFO:teuthology.orchestra.run.smithi076.stdout:5494 read 0x2b2cd81 thru 0x2b2f418 (0x2698 bytes) 2024-09-14T22:54:35.798 INFO:teuthology.orchestra.run.smithi076.stdout:5495 clone 42 order 25 su 16777216 sc 11 2024-09-14T22:54:35.828 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:35.828 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:35.828 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:35.828 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:35.828 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:35.837 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:35.878 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:35.877+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:35.969 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:35.968+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:35.981 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:35.981 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:35.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:35.981 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:35.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.981 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:35.981 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.129482780000004, 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:35.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:35.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:35.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.985 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.128545189999997, 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:35.986 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:54:35.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:35.990 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.128453827000001, 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:35.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:54:35.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:35.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.127808942000001, 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:35.995 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:35.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:54:35.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:35.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.127321562999995, 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.000 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.126434287999999, 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.004 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:54:36.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.008 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.125841863999995, 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:54:36.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.125109047999999, 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.013 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.017 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.125009899000005, 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:54:36.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:54:36.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.124691440999996, 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.022 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:54:36.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.124181946999997, 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.027 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:54:36.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.124124088000002, 2024-09-14T22:54:36.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:36.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:54:36.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.123331465000007, 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.036 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.037 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:54:36.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.122656026000001, 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.041 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:54:36.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.122522716000006, 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.045 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:54:36.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.121905792999996, 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.050 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:54:36.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.053 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.121838936000003, 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.054 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.058 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.121388124999996, 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.059 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:36.060 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.061 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.062 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.121267892000006, 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.063 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.064 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.065 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.066 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.067 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 72.119949765000001, 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.068 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:54:36.069 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.070 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.071 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.072 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:54:36.072 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:54:36.072 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.072 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.072 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.072 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.072 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.072 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:36.073 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:36.074 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:36.072+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:36.174 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:36.174+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:36.177 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:36.178 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:36.178 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:36.178 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:36.186 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:36.233 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:54:36.275 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:36.274+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:36.299 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:36.300 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:36.310 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:36.376 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:36.375+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:36.440 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.441147938, 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:54:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:54:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:54:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.441128900000002, 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:54:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:54:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.441092943999998, 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:54:36.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.456 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.441071823999998, 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:54:36.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.460 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.439864909999997, 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.461 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:54:36.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:54:36.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.438035462999999, 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.465 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:36.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:54:36.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:54:36.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.437481175999999, 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:54:36.469 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:54:36.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:54:36.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.435496458000003, 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.474 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:54:36.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.435144137999998, 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.478 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:54:36.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:54:36.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.434860694999998, 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.483 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:36.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.487 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.434572598000003, 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:54:36.491 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.434197769999997, 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.492 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.433876955000002, 2024-09-14T22:54:36.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.420191719999998, 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.501 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:54:36.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.419856910999997, 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.506 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:36.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:54:36.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:54:36.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:36.511 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.419526748999999, 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:54:36.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:54:36.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:36.516 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.419305962999999, 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:54:36.519 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.419036662000003, 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.521 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.523 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.525 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.418849565999999, 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:54:36.528 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.529 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.418531407000003, 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:36.530 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:36.531 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.532 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.533 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:54:36.534 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:54:36.534 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.534 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.534 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.534 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.534 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.534 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:36.536 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:5496 writesame 0x5cb8e1e thru 0x5cbc281 data_size 0xe(0x3464 bytes) 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:5497 write 0x87584bc thru 0x8758890 (0x3d5 bytes) 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:5500 read 0x8878998 thru 0x887eced (0x6356 bytes) 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:5501 read 0xb4678d8 thru 0xb472ffc (0xb725 bytes) 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:5503 read 0x66866a thru 0x66f020 (0x69b7 bytes) 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:5504 read 0x51159bf thru 0x511eddf (0x9421 bytes) 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:5506 writesame 0x8754e5d thru 0x876151b data_size 0x1(0xc6bf bytes) 2024-09-14T22:54:36.537 INFO:teuthology.orchestra.run.smithi076.stdout:5507 read 0xe81a7f2 thru 0xe824c51 (0xa460 bytes) 2024-09-14T22:54:36.538 INFO:teuthology.orchestra.run.smithi076.stdout:5509 write 0xe06be7d thru 0xe07ab79 (0xecfd bytes) 2024-09-14T22:54:36.538 INFO:teuthology.orchestra.run.smithi076.stdout:5510 write 0x95e66e6 thru 0x95ea001 (0x391c bytes) 2024-09-14T22:54:36.538 INFO:teuthology.orchestra.run.smithi076.stdout:5511 trunc from 0xec06ff8 to 0x82b0c78 2024-09-14T22:54:36.538 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:36.534+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:36.544 INFO:teuthology.orchestra.run.smithi076.stdout:5515 read 0x96ba1 thru 0xa621a (0xf67a bytes) 2024-09-14T22:54:36.550 INFO:teuthology.orchestra.run.smithi076.stdout:5517 trunc from 0x82b0c78 to 0xd04b8f1 2024-09-14T22:54:36.556 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.556 INFO:teuthology.orchestra.run.smithi076.stdout:5519 writesame 0xb4ebd03 thru 0xb4f2fe5 data_size 0x1(0x72e3 bytes) 2024-09-14T22:54:36.574 INFO:teuthology.orchestra.run.smithi076.stdout:5520 punch from 0x4f47ffc to 0x4f560de, (0xe0e2 bytes) 2024-09-14T22:54:36.584 INFO:teuthology.orchestra.run.smithi076.stdout:5525 trunc from 0xd04b8f1 to 0x4670119 2024-09-14T22:54:36.597 INFO:teuthology.orchestra.run.smithi076.stdout:5527 write 0x34d9e8b thru 0x34dfac6 (0x5c3c bytes) 2024-09-14T22:54:36.598 INFO:teuthology.orchestra.run.smithi076.stdout:5528 read 0x2a6e352 thru 0x2a73460 (0x510f bytes) 2024-09-14T22:54:36.610 INFO:teuthology.orchestra.run.smithi076.stdout:5529 write 0x9100e0f thru 0x910fe4f (0xf041 bytes) 2024-09-14T22:54:36.613 INFO:teuthology.orchestra.run.smithi076.stdout:5530 writesame 0x86e30bf thru 0x86ec6dd data_size 0x1(0x961f bytes) 2024-09-14T22:54:36.635 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:36.634+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:36.642 INFO:teuthology.orchestra.run.smithi076.stdout:5531 trunc from 0x910fe50 to 0x5b9f544 2024-09-14T22:54:36.656 INFO:teuthology.orchestra.run.smithi076.stdout:5533 write 0x39dbdf4 thru 0x39e8796 (0xc9a3 bytes) 2024-09-14T22:54:36.658 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.663 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:36.663 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:36.663 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:36.663 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:36.674 INFO:teuthology.orchestra.run.smithi076.stdout:5535 writesame 0xe9834a7 thru 0xe98a185 data_size 0x1(0x6cdf bytes) 2024-09-14T22:54:36.675 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:36.676 INFO:teuthology.orchestra.run.smithi076.stdout:5537 punch from 0x111e1b7 to 0x112c5b9, (0xe402 bytes) 2024-09-14T22:54:36.677 INFO:teuthology.orchestra.run.smithi076.stdout:5538 read 0x666b082 thru 0x66752c5 (0xa244 bytes) 2024-09-14T22:54:36.677 INFO:teuthology.orchestra.run.smithi076.stdout:5539 read 0x85b7b51 thru 0x85bbb0a (0x3fba bytes) 2024-09-14T22:54:36.677 INFO:teuthology.orchestra.run.smithi076.stdout:5540 write 0x966e359 thru 0x9670eaa (0x2b52 bytes) 2024-09-14T22:54:36.679 INFO:teuthology.orchestra.run.smithi076.stdout:5541 read 0x32590e2 thru 0x3267872 (0xe791 bytes) 2024-09-14T22:54:36.679 INFO:teuthology.orchestra.run.smithi076.stdout:5542 read 0xce38769 thru 0xce3e735 (0x5fcd bytes) 2024-09-14T22:54:36.685 INFO:teuthology.orchestra.run.smithi076.stdout:5543 trunc from 0xe98a186 to 0xa63ca98 2024-09-14T22:54:36.692 INFO:teuthology.orchestra.run.smithi076.stdout:5544 write 0x2d295f5 thru 0x2d2e38c (0x4d98 bytes) 2024-09-14T22:54:36.693 INFO:teuthology.orchestra.run.smithi076.stdout:5545 read 0x64e2131 thru 0x64e271f (0x5ef bytes) 2024-09-14T22:54:36.702 INFO:teuthology.orchestra.run.smithi076.stdout:5546 trunc from 0xa63ca98 to 0x2b1fff0 2024-09-14T22:54:36.714 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.715 INFO:teuthology.orchestra.run.smithi076.stdout:5549 read 0x1d44f8e thru 0x1d46372 (0x13e5 bytes) 2024-09-14T22:54:36.715 INFO:teuthology.orchestra.run.smithi076.stdout:5550 read 0x79edc4 thru 0x7acbbf (0xddfc bytes) 2024-09-14T22:54:36.715 INFO:teuthology.orchestra.run.smithi076.stdout:5551 read 0x65e51c thru 0x66d1fd (0xece2 bytes) 2024-09-14T22:54:36.716 INFO:teuthology.orchestra.run.smithi076.stdout:5552 read 0x19a2262 thru 0x19a38d9 (0x1678 bytes) 2024-09-14T22:54:36.726 INFO:teuthology.orchestra.run.smithi076.stdout:5553 writesame 0x5d69ea8 thru 0x5d6dc36 data_size 0x11(0x3d8f bytes) 2024-09-14T22:54:36.726 INFO:teuthology.orchestra.run.smithi076.stdout:5555 read 0x2c8f41b thru 0x2c99636 (0xa21c bytes) 2024-09-14T22:54:36.728 INFO:teuthology.orchestra.run.smithi076.stdout:5557 trunc from 0x5d6dc37 to 0x758ebb6 2024-09-14T22:54:36.737 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:36.735+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:36.744 INFO:teuthology.orchestra.run.smithi076.stdout:5558 write 0xa84424d thru 0xa8516ea (0xd49e bytes) 2024-09-14T22:54:36.754 INFO:teuthology.orchestra.run.smithi076.stdout:5559 write 0xb059d69 thru 0xb063d66 (0x9ffe bytes) 2024-09-14T22:54:36.755 INFO:teuthology.orchestra.run.smithi076.stdout:5560 punch from 0x1e42b0 to 0x1e4b93, (0x8e3 bytes) 2024-09-14T22:54:36.755 INFO:teuthology.orchestra.run.smithi076.stdout:5562 punch from 0x829e8ea to 0x82a8db5, (0xa4cb bytes) 2024-09-14T22:54:36.755 INFO:teuthology.orchestra.run.smithi076.stdout:5564 write 0x4f3a3f1 thru 0x4f42c69 (0x8879 bytes) 2024-09-14T22:54:36.762 INFO:teuthology.orchestra.run.smithi076.stdout:5565 trunc from 0xb063d67 to 0x88e4a06 2024-09-14T22:54:36.772 INFO:teuthology.orchestra.run.smithi076.stdout:5567 punch from 0x1e2629c to 0x1e30d1d, (0xaa81 bytes) 2024-09-14T22:54:36.773 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.780 INFO:teuthology.orchestra.run.smithi076.stdout:5571 write 0xa343d40 thru 0xa34d6c4 (0x9985 bytes) 2024-09-14T22:54:36.784 INFO:teuthology.orchestra.run.smithi076.stdout:5572 read 0x36ba8ef thru 0x36c1244 (0x6956 bytes) 2024-09-14T22:54:36.793 INFO:teuthology.orchestra.run.smithi076.stdout:5573 writesame 0xcf5a2f7 thru 0xcf5c2ae data_size 0x28(0x1fb8 bytes) 2024-09-14T22:54:36.794 INFO:teuthology.orchestra.run.smithi076.stdout:5575 writesame 0x79341b3 thru 0x79380ad data_size 0x1(0x3efb bytes) 2024-09-14T22:54:36.797 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:36.797 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:36.806 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.806 INFO:teuthology.orchestra.run.smithi076.stdout:5578 write 0x8300371 thru 0x8309429 (0x90b9 bytes) 2024-09-14T22:54:36.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:36.809 INFO:teuthology.orchestra.run.smithi076.stdout:5579 punch from 0x7b829c7 to 0x7b8ad1c, (0x8355 bytes) 2024-09-14T22:54:36.810 INFO:teuthology.orchestra.run.smithi076.stdout:5580 write 0x6f143b1 thru 0x6f167b7 (0x2407 bytes) 2024-09-14T22:54:36.812 INFO:teuthology.orchestra.run.smithi076.stdout:5581 read 0x633068b thru 0x633248a (0x1e00 bytes) 2024-09-14T22:54:36.812 INFO:teuthology.orchestra.run.smithi076.stdout:5582 read 0x6399ac7 thru 0x639dc6c (0x41a6 bytes) 2024-09-14T22:54:36.813 INFO:teuthology.orchestra.run.smithi076.stdout:5584 read 0x70f3b9 thru 0x71cea3 (0xdaeb bytes) 2024-09-14T22:54:36.814 INFO:teuthology.orchestra.run.smithi076.stdout:5585 trunc from 0xcf5c2af to 0xbe13803 2024-09-14T22:54:36.822 INFO:teuthology.orchestra.run.smithi076.stdout:5586 write 0x81f7c40 thru 0x8204a3a (0xcdfb bytes) 2024-09-14T22:54:36.823 INFO:teuthology.orchestra.run.smithi076.stdout:5587 read 0x8cd19c9 thru 0x8cd293b (0xf73 bytes) 2024-09-14T22:54:36.824 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.824 INFO:teuthology.orchestra.run.smithi076.stdout:5590 read 0xb3d548d thru 0xb3e05db (0xb14f bytes) 2024-09-14T22:54:36.824 INFO:teuthology.orchestra.run.smithi076.stdout:5591 read 0x839cbde thru 0x83ab1f7 (0xe61a bytes) 2024-09-14T22:54:36.828 INFO:teuthology.orchestra.run.smithi076.stdout:5592 trunc from 0xbe13803 to 0xe49ece9 2024-09-14T22:54:36.833 INFO:teuthology.orchestra.run.smithi076.stdout:5593 punch from 0xb93088d to 0xb932baa, (0x231d bytes) 2024-09-14T22:54:36.834 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.834 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.838 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:36.836+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:36.838 INFO:teuthology.orchestra.run.smithi076.stdout:5596 write 0xe7bc090 thru 0xe7bf055 (0x2fc6 bytes) 2024-09-14T22:54:36.841 INFO:teuthology.orchestra.run.smithi076.stdout:5597 read 0x432b057 thru 0x433aae8 (0xfa92 bytes) 2024-09-14T22:54:36.855 INFO:teuthology.orchestra.run.smithi076.stdout:5599 trunc from 0xe7bf056 to 0x38f6173 2024-09-14T22:54:36.869 INFO:teuthology.orchestra.run.smithi076.stdout:5600 writesame 0x318dddf thru 0x318ef9f data_size 0xf(0x11c1 bytes) 2024-09-14T22:54:36.892 INFO:teuthology.orchestra.run.smithi076.stdout:5601 writesame 0xed68255 thru 0xed72b97 data_size 0x1(0xa943 bytes) 2024-09-14T22:54:36.893 INFO:teuthology.orchestra.run.smithi076.stdout:5603 read 0x9e1a0c1 thru 0x9e25a2c (0xb96c bytes) 2024-09-14T22:54:36.893 INFO:teuthology.orchestra.run.smithi076.stdout:5604 read 0x5a7cdbb thru 0x5a8370a (0x6950 bytes) 2024-09-14T22:54:36.893 INFO:teuthology.orchestra.run.smithi076.stdout:5605 writesame 0x74aafb0 thru 0x74b75b0 data_size 0x1(0xc601 bytes) 2024-09-14T22:54:36.919 INFO:teuthology.orchestra.run.smithi076.stdout:5606 write 0x21bafab thru 0x21c4006 (0x905c bytes) 2024-09-14T22:54:36.922 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.922 INFO:teuthology.orchestra.run.smithi076.stdout:5609 writesame 0x54bf294 thru 0x54ccac0 data_size 0x81(0xd82d bytes) 2024-09-14T22:54:36.925 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:36.926 INFO:teuthology.orchestra.run.smithi076.stdout:5611 writesame 0xab226f2 thru 0xab29091 data_size 0x82(0x69a0 bytes) 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 69.717677515999995, 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.929 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:54:36.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:54:36.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.594015742000003, 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.934 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:54:36.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:54:36.938 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 47.662248271000003, 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.943 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 22.752302028999999, 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:54:36.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172324 3.4 3:213525dd:::rbd_data.113598f99d1a.00000000000000b5:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 22.739512018999999, 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051499340000000001, 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172324 2024-09-14T22:54:36.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186688+0000", 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1829999999999999e-06 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186693+0000", 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4100000000000001e-06 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186705+0000", 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2169e-05 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186712+0000", 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9789999999999996e-06 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186758+0000", 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6687000000000002e-05 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186839+0000", 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-05 2024-09-14T22:54:36.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186980+0000", 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000141569 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191467+0000", 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0044866840000000003 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191734+0000", 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026690299999999998 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191793+0000", 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9447000000000002e-05 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191837+0000", 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3433000000000002e-05 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.954 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 13.803641386000001, 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.955 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:54:36.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:54:36.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:36.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:54:36.960 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.347564509, 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.961 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:54:36.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.965 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.347537187, 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.347125695999999, 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:54:36.970 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:54:36.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.345466429, 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.974 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:54:36.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:54:36.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:36.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:54:36.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.341435004999999, 2024-09-14T22:54:36.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:54:36.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:54:36.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194089 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:36.984 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.340847339, 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005182201, 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194089 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:36.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585353+0000", 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8899999999999998e-07 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585357+0000", 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8470000000000003e-06 2024-09-14T22:54:36.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585369+0000", 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1992e-05 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585377+0000", 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6090000000000003e-06 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590237+0000", 2024-09-14T22:54:36.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048596910000000002 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590352+0000", 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000115387 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590534+0000", 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018168599999999999 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.988 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194119 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.339611979000001, 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051579030000000001, 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.989 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194119 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8200000000000001e-07 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586590+0000", 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586596+0000", 2024-09-14T22:54:36.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.212e-06 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586601+0000", 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8289999999999998e-06 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591330+0000", 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047292439999999996 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591418+0000", 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8448000000000004e-05 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591745+0000", 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00032651799999999998 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:36.993 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.339591529, 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:36.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:54:36.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.339571854999999, 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:36.998 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:36.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:54:37.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.002 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.339533201, 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.003 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.006 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.339513321, 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.007 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:54:37.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:37.011 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.966342128, 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:37.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:54:37.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:37.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:54:37.017 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.965340834, 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:54:37.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:54:37.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:37.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:54:37.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.6592846430000003, 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.024 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.029 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.030 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.030 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:37.031 INFO:teuthology.orchestra.run.smithi076.stdout:5614 write 0x1250a3d thru 0x1259fb6 (0x957a bytes) 2024-09-14T22:54:37.031 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.031 INFO:teuthology.orchestra.run.smithi076.stdout:5617 punch from 0xccaeaae to 0xccb1303, (0x2855 bytes) 2024-09-14T22:54:37.031 INFO:teuthology.orchestra.run.smithi076.stdout:5618 write 0xc7c973d thru 0xc7d43cc (0xac90 bytes) 2024-09-14T22:54:37.031 INFO:teuthology.orchestra.run.smithi076.stdout:5619 writesame 0x2b9fc2d thru 0x2bad979 data_size 0x1(0xdd4d bytes) 2024-09-14T22:54:37.032 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:37.033 INFO:teuthology.orchestra.run.smithi076.stdout:5620 trunc from 0xed72b98 to 0x280bcca 2024-09-14T22:54:37.033 INFO:teuthology.orchestra.run.smithi076.stdout:5621 punch from 0xbe174f to 0xbe943a, (0x7ceb bytes) 2024-09-14T22:54:37.034 INFO:teuthology.orchestra.run.smithi076.stdout:5622 read 0x138ddad thru 0x1396f3c (0x9190 bytes) 2024-09-14T22:54:37.034 INFO:teuthology.orchestra.run.smithi076.stdout:5623 write 0xb49fbfc thru 0xb4a1906 (0x1d0b bytes) 2024-09-14T22:54:37.034 INFO:teuthology.orchestra.run.smithi076.stdout:5624 write 0x3e184de thru 0x3e1a431 (0x1f54 bytes) 2024-09-14T22:54:37.034 INFO:teuthology.orchestra.run.smithi076.stdout:5625 trunc from 0xb4a1907 to 0x64c85cb 2024-09-14T22:54:37.034 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:37.030+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:37.034 INFO:teuthology.orchestra.run.smithi076.stdout:5626 read 0x4cdbb89 thru 0x4cdfece (0x4346 bytes) 2024-09-14T22:54:37.042 INFO:teuthology.orchestra.run.smithi076.stdout:5627 trunc from 0x64c85cb to 0xd51b63e 2024-09-14T22:54:37.047 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.048 INFO:teuthology.orchestra.run.smithi076.stdout:5629 write 0x5b7b760 thru 0x5b81789 (0x602a bytes) 2024-09-14T22:54:37.050 INFO:teuthology.orchestra.run.smithi076.stdout:5631 write 0x9c502eb thru 0x9c5d663 (0xd379 bytes) 2024-09-14T22:54:37.053 INFO:teuthology.orchestra.run.smithi076.stdout:5632 read 0xcd6772b thru 0xcd70560 (0x8e36 bytes) 2024-09-14T22:54:37.053 INFO:teuthology.orchestra.run.smithi076.stdout:5636 write 0x6fb6984 thru 0x6fb6c4e (0x2cb bytes) 2024-09-14T22:54:37.056 INFO:teuthology.orchestra.run.smithi076.stdout:5639 punch from 0x1e9f94f to 0x1eaee32, (0xf4e3 bytes) 2024-09-14T22:54:37.056 INFO:teuthology.orchestra.run.smithi076.stdout:5640 read 0x499b059 thru 0x49a7551 (0xc4f9 bytes) 2024-09-14T22:54:37.060 INFO:teuthology.orchestra.run.smithi076.stdout:5641 trunc from 0xd51b63e to 0xa089b37 2024-09-14T22:54:37.071 INFO:teuthology.orchestra.run.smithi076.stdout:5642 trunc from 0xa089b37 to 0x66df9b6 2024-09-14T22:54:37.085 INFO:teuthology.orchestra.run.smithi076.stdout:5643 trunc from 0x66df9b6 to 0xb0bdea9 2024-09-14T22:54:37.090 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.097 INFO:teuthology.orchestra.run.smithi076.stdout:5645 trunc from 0xb0bdea9 to 0x5784be7 2024-09-14T22:54:37.106 INFO:teuthology.orchestra.run.smithi076.stdout:5647 read 0x3e3e73a thru 0x3e40c52 (0x2519 bytes) 2024-09-14T22:54:37.113 INFO:teuthology.orchestra.run.smithi076.stdout:5649 writesame 0x653f5e7 thru 0x6540b81 data_size 0x1(0x159b bytes) 2024-09-14T22:54:37.113 INFO:teuthology.orchestra.run.smithi076.stdout:5650 write 0xb1d6f7 thru 0xb1ed8b (0x1695 bytes) 2024-09-14T22:54:37.115 INFO:teuthology.orchestra.run.smithi076.stdout:5651 punch from 0x1e3ee1a to 0x1e4d23a, (0xe420 bytes) 2024-09-14T22:54:37.115 INFO:teuthology.orchestra.run.smithi076.stdout:5652 punch from 0x2185478 to 0x21914e7, (0xc06f bytes) 2024-09-14T22:54:37.115 INFO:teuthology.orchestra.run.smithi076.stdout:5655 punch from 0x2e948ef to 0x2ea24cf, (0xdbe0 bytes) 2024-09-14T22:54:37.115 INFO:teuthology.orchestra.run.smithi076.stdout:5656 read 0x4c48e69 thru 0x4c57eac (0xf044 bytes) 2024-09-14T22:54:37.115 INFO:teuthology.orchestra.run.smithi076.stdout:5657 read 0x4e1299a thru 0x4e19aa3 (0x710a bytes) 2024-09-14T22:54:37.124 INFO:teuthology.orchestra.run.smithi076.stdout:5658 trunc from 0x6540b82 to 0xe47adaa 2024-09-14T22:54:37.129 INFO:teuthology.orchestra.run.smithi076.stdout:5659 read 0x3585d6e thru 0x358e2bd (0x8550 bytes) 2024-09-14T22:54:37.130 INFO:teuthology.orchestra.run.smithi076.stdout:5660 write 0xc9a1073 thru 0xc9a75ad (0x653b bytes) 2024-09-14T22:54:37.132 INFO:teuthology.orchestra.run.smithi076.stdout:5661 writesame 0xc21131c thru 0xc2189b6 data_size 0x1d(0x769b bytes) 2024-09-14T22:54:37.132 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:37.131+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:37.134 INFO:teuthology.orchestra.run.smithi076.stdout:5663 read 0x5651b1c thru 0x565fa8b (0xdf70 bytes) 2024-09-14T22:54:37.134 INFO:teuthology.orchestra.run.smithi076.stdout:5665 writesame 0x282ce6b thru 0x2834213 data_size 0x1(0x73a9 bytes) 2024-09-14T22:54:37.150 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:37.150 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:37.150 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:37.150 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:37.160 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:37.160 INFO:teuthology.orchestra.run.smithi076.stdout:5666 write 0xe991534 thru 0xe993c63 (0x2730 bytes) 2024-09-14T22:54:37.161 INFO:teuthology.orchestra.run.smithi076.stdout:5667 writesame 0xac03636 thru 0xac0a02a data_size 0x7(0x69f5 bytes) 2024-09-14T22:54:37.165 INFO:teuthology.orchestra.run.smithi076.stdout:5668 write 0x33ebf4f thru 0x33ee814 (0x28c6 bytes) 2024-09-14T22:54:37.167 INFO:teuthology.orchestra.run.smithi076.stdout:5669 write 0x840e3a1 thru 0x841c734 (0xe394 bytes) 2024-09-14T22:54:37.170 INFO:teuthology.orchestra.run.smithi076.stdout:5671 read 0x6df0c14 thru 0x6df560e (0x49fb bytes) 2024-09-14T22:54:37.170 INFO:teuthology.orchestra.run.smithi076.stdout:5674 read 0xd41a481 thru 0xd41dac6 (0x3646 bytes) 2024-09-14T22:54:37.171 INFO:teuthology.orchestra.run.smithi076.stdout:5675 punch from 0xc8c52b9 to 0xc8c5953, (0x69a bytes) 2024-09-14T22:54:37.171 INFO:teuthology.orchestra.run.smithi076.stdout:5676 clone 43 order 23 su 1048576 sc 14 2024-09-14T22:54:37.230 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:54:37.233 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:37.232+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:37.274 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:37.274 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:37.283 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:37.286 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone42 from 0xe993c64 (overlap 0x280bcca) to 0x0 2024-09-14T22:54:37.332 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-14T22:54:37.334 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:37.333+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:37.410 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:37.410 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503866017000007, 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:54:37.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.415 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503845480999999, 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:54:37.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503815481999993, 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.420 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:37.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:54:37.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.424 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503784328999998, 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:54:37.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503761673, 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:37.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.433 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503741172000005, 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:54:37.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503721411000001, 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:54:37.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503700625999997, 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.443 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:37.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.447 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503668214000001, 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.448 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:37.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.452 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503646814999996, 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:37.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503585415999993, 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:54:37.457 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:54:37.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.461 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503545502999998, 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.462 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:54:37.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:37.466 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503515746999994, 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:37.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:54:37.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503485511999997, 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.471 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:54:37.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.475 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503460773, 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.476 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:54:37.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.480 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503437343000002, 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503341606999996, 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.485 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:54:37.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503312684999997, 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:54:37.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:54:37.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 75.503289359999997, 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:37.494 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:37.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.498 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 52.370591617000002, 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:54:37.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:37.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:37.503 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:37.505 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:37.505 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:37.504+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:37.605 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:37.604+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:37.610 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:37.610 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:37.611 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:37.611 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:37.619 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:37.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:37.637 INFO:teuthology.orchestra.run.smithi076.stdout:5677 read 0xb5de28e thru 0xb5eadce (0xcb41 bytes) 2024-09-14T22:54:37.638 INFO:teuthology.orchestra.run.smithi076.stdout:5678 write 0xd0e2d73 thru 0xd0efdd2 (0xd060 bytes) 2024-09-14T22:54:37.646 INFO:teuthology.orchestra.run.smithi076.stdout:5679 writesame 0xb4f74ad thru 0xb4f9be0 data_size 0x1a(0x2734 bytes) 2024-09-14T22:54:37.647 INFO:teuthology.orchestra.run.smithi076.stdout:5680 write 0xc2c133e thru 0xc2cb509 (0xa1cc bytes) 2024-09-14T22:54:37.649 INFO:teuthology.orchestra.run.smithi076.stdout:5681 write 0xd826be0 thru 0xd8318a2 (0xacc3 bytes) 2024-09-14T22:54:37.658 INFO:teuthology.orchestra.run.smithi076.stdout:5683 write 0xece7c86 thru 0xece8812 (0xb8d bytes) 2024-09-14T22:54:37.661 INFO:teuthology.orchestra.run.smithi076.stdout:5684 write 0x834b5a5 thru 0x83566e6 (0xb142 bytes) 2024-09-14T22:54:37.665 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.665 INFO:teuthology.orchestra.run.smithi076.stdout:5686 write 0x6d0c593 thru 0x6d1095d (0x43cb bytes) 2024-09-14T22:54:37.669 INFO:teuthology.orchestra.run.smithi076.stdout:5687 write 0x64c3437 thru 0x64c6b38 (0x3702 bytes) 2024-09-14T22:54:37.673 INFO:teuthology.orchestra.run.smithi076.stdout:5689 writesame 0xc6df422 thru 0xc6eb515 data_size 0x35(0xc0f4 bytes) 2024-09-14T22:54:37.677 INFO:teuthology.orchestra.run.smithi076.stdout:5690 read 0xef3965 thru 0xefb3c2 (0x7a5e bytes) 2024-09-14T22:54:37.690 INFO:teuthology.orchestra.run.smithi076.stdout:5691 trunc from 0xece8813 to 0x4660b7b 2024-09-14T22:54:37.706 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:37.705+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:37.722 INFO:teuthology.orchestra.run.smithi076.stdout:5692 read 0x1797c85 thru 0x1799ded (0x2169 bytes) 2024-09-14T22:54:37.722 INFO:teuthology.orchestra.run.smithi076.stdout:5694 read 0x31326c4 thru 0x313d381 (0xacbe bytes) 2024-09-14T22:54:37.733 INFO:teuthology.orchestra.run.smithi076.stdout:5695 write 0x815118a thru 0x81591eb (0x8062 bytes) 2024-09-14T22:54:37.738 INFO:teuthology.orchestra.run.smithi076.stdout:5696 trunc from 0x81591ec to 0x670ae06 2024-09-14T22:54:37.751 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.751 INFO:teuthology.orchestra.run.smithi076.stdout:5699 write 0x4fdf4de thru 0x4fe7de0 (0x8903 bytes) 2024-09-14T22:54:37.752 INFO:teuthology.orchestra.run.smithi076.stdout:5701 write 0x282804 thru 0x2855c4 (0x2dc1 bytes) 2024-09-14T22:54:37.754 INFO:teuthology.orchestra.run.smithi076.stdout:5702 trunc from 0x670ae06 to 0x60e4c41 2024-09-14T22:54:37.761 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.761 INFO:teuthology.orchestra.run.smithi076.stdout:5704 write 0x4ff8c94 thru 0x4ffe72d (0x5a9a bytes) 2024-09-14T22:54:37.763 INFO:teuthology.orchestra.run.smithi076.stdout:5707 read 0xf7ff4c thru 0xf8b05b (0xb110 bytes) 2024-09-14T22:54:37.763 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:37.763 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:37.771 INFO:teuthology.orchestra.run.smithi076.stdout:5708 writesame 0x86e14c5 thru 0x86e9edd data_size 0x1(0x8a19 bytes) 2024-09-14T22:54:37.771 INFO:teuthology.orchestra.run.smithi076.stdout:5709 read 0x66bb5d4 thru 0x66c8f4c (0xd979 bytes) 2024-09-14T22:54:37.772 INFO:teuthology.orchestra.run.smithi076.stdout:5710 punch from 0x37907c to 0x380b10, (0x7a94 bytes) 2024-09-14T22:54:37.779 INFO:teuthology.orchestra.run.smithi076.stdout:5711 trunc from 0x86e9ede to 0xed117c4 2024-09-14T22:54:37.780 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:37.780 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:37.782 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:37.782 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:37.789 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:37.799 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:37.799 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:37.799 INFO:teuthology.orchestra.run.smithi076.stdout:5712 trunc from 0xed117c4 to 0x2c491e0 2024-09-14T22:54:37.817 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:37.817 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:37.824 INFO:teuthology.orchestra.run.smithi076.stdout:5713 writesame 0xace0a82 thru 0xace93df data_size 0x3(0x895e bytes) 2024-09-14T22:54:37.824 INFO:teuthology.orchestra.run.smithi076.stdout:5714 writesame 0x6f4d43a thru 0x6f5d415 data_size 0x83(0xffdc bytes) 2024-09-14T22:54:37.826 INFO:teuthology.orchestra.run.smithi076.stdout:5717 writesame 0x14ec614 thru 0x14f8c30 data_size 0x1(0xc61d bytes) 2024-09-14T22:54:37.833 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:37.832+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:37.834 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:37.834 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:37.850 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:37.851 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:37.860 INFO:teuthology.orchestra.run.smithi076.stdout:5718 trunc from 0xace93e0 to 0xc33ef50 2024-09-14T22:54:37.866 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.867 INFO:teuthology.orchestra.run.smithi076.stdout:5720 read 0xb09fd06 thru 0xb0a8ac5 (0x8dc0 bytes) 2024-09-14T22:54:37.867 INFO:teuthology.orchestra.run.smithi076.stdout:5721 read 0x48d55d0 thru 0x48d77c2 (0x21f3 bytes) 2024-09-14T22:54:37.867 INFO:teuthology.orchestra.run.smithi076.stdout:5722 writesame 0x6f137c thru 0x6fb961 data_size 0x3e(0xa5e6 bytes) 2024-09-14T22:54:37.868 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:37.868 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:37.869 INFO:teuthology.orchestra.run.smithi076.stdout:5724 writesame 0x5819c9f thru 0x5825fc5 data_size 0x5b(0xc327 bytes) 2024-09-14T22:54:37.871 INFO:teuthology.orchestra.run.smithi076.stdout:5725 write 0x6b6c082 thru 0x6b7a88c (0xe80b bytes) 2024-09-14T22:54:37.874 INFO:teuthology.orchestra.run.smithi076.stdout:5726 trunc from 0xc33ef50 to 0xdb10c0a 2024-09-14T22:54:37.880 INFO:teuthology.orchestra.run.smithi076.stdout:5727 read 0x54d6ccf thru 0x54e015a (0x948c bytes) 2024-09-14T22:54:37.881 INFO:teuthology.orchestra.run.smithi076.stdout:5728 read 0x45d56f2 thru 0x45de5e5 (0x8ef4 bytes) 2024-09-14T22:54:37.881 INFO:teuthology.orchestra.run.smithi076.stdout:5729 write 0xcaac1a2 thru 0xcab8510 (0xc36f bytes) 2024-09-14T22:54:37.884 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.885 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:37.885 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:37.893 INFO:teuthology.orchestra.run.smithi076.stdout:5731 trunc from 0xdb10c0a to 0x6313158 2024-09-14T22:54:37.897 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:37.897 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:37.908 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:37.909 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.909 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:37.907+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:37.918 INFO:teuthology.orchestra.run.smithi076.stdout:5735 write 0x9e7982b thru 0x9e7c1f3 (0x29c9 bytes) 2024-09-14T22:54:37.922 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.922 INFO:teuthology.orchestra.run.smithi076.stdout:5738 write 0x8a368ba thru 0x8a45fe4 (0xf72b bytes) 2024-09-14T22:54:37.925 INFO:teuthology.orchestra.run.smithi076.stdout:5739 write 0x7eace6b thru 0x7ebcbaa (0xfd40 bytes) 2024-09-14T22:54:37.928 INFO:teuthology.orchestra.run.smithi076.stdout:5740 read 0x9d07b61 thru 0x9d153d8 (0xd878 bytes) 2024-09-14T22:54:37.928 INFO:teuthology.orchestra.run.smithi076.stdout:5741 write 0x53ce561 thru 0x53dd125 (0xebc5 bytes) 2024-09-14T22:54:37.930 INFO:teuthology.orchestra.run.smithi076.stdout:5742 read 0x7013b51 thru 0x701933b (0x57eb bytes) 2024-09-14T22:54:37.930 INFO:teuthology.orchestra.run.smithi076.stdout:5744 writesame 0x5621e22 thru 0x5622cfa data_size 0x7(0xed9 bytes) 2024-09-14T22:54:37.932 INFO:teuthology.orchestra.run.smithi076.stdout:5745 writesame 0x826544 thru 0x82b10f data_size 0x54(0x4bcc bytes) 2024-09-14T22:54:37.933 INFO:teuthology.orchestra.run.smithi076.stdout:5746 punch from 0x12e2aca to 0x12e3558, (0xa8e bytes) 2024-09-14T22:54:37.933 INFO:teuthology.orchestra.run.smithi076.stdout:5747 read 0x50869 thru 0x5a998 (0xa130 bytes) 2024-09-14T22:54:37.934 INFO:teuthology.orchestra.run.smithi076.stdout:5748 read 0x55775ca thru 0x557d8cd (0x6304 bytes) 2024-09-14T22:54:37.934 INFO:teuthology.orchestra.run.smithi076.stdout:5749 read 0x6a2490e thru 0x6a329e8 (0xe0db bytes) 2024-09-14T22:54:37.934 INFO:teuthology.orchestra.run.smithi076.stdout:5750 write 0xf9a97c thru 0xf9d6f1 (0x2d76 bytes) 2024-09-14T22:54:37.936 INFO:teuthology.orchestra.run.smithi076.stdout:5751 read 0x1d39a88 thru 0x1d3bd5a (0x22d3 bytes) 2024-09-14T22:54:37.936 INFO:teuthology.orchestra.run.smithi076.stdout:5753 read 0x67b0cd2 thru 0x67b9ce7 (0x9016 bytes) 2024-09-14T22:54:37.937 INFO:teuthology.orchestra.run.smithi076.stdout:5754 read 0xe8e04b thru 0xe9b2c8 (0xd27e bytes) 2024-09-14T22:54:37.940 INFO:teuthology.orchestra.run.smithi076.stdout:5755 trunc from 0x9e7c1f4 to 0xca65093 2024-09-14T22:54:37.946 INFO:teuthology.orchestra.run.smithi076.stdout:5756 write 0x53995ab thru 0x539c825 (0x327b bytes) 2024-09-14T22:54:37.947 INFO:teuthology.orchestra.run.smithi076.stdout:5757 read 0x5158782 thru 0x515ca41 (0x42c0 bytes) 2024-09-14T22:54:37.948 INFO:teuthology.orchestra.run.smithi076.stdout:5759 write 0xb1976ee thru 0xb1a2868 (0xb17b bytes) 2024-09-14T22:54:37.953 INFO:teuthology.orchestra.run.smithi076.stdout:5760 trunc from 0xca65093 to 0xe588a4a 2024-09-14T22:54:37.960 INFO:teuthology.orchestra.run.smithi076.stdout:5761 writesame 0x3cb7bf4 thru 0x3cbc373 data_size 0x40(0x4780 bytes) 2024-09-14T22:54:37.961 INFO:teuthology.orchestra.run.smithi076.stdout:5762 write 0x87c8fab thru 0x87cb3ac (0x2402 bytes) 2024-09-14T22:54:37.963 INFO:teuthology.orchestra.run.smithi076.stdout:5764 write 0x429da8 thru 0x438bd4 (0xee2d bytes) 2024-09-14T22:54:37.964 INFO:teuthology.orchestra.run.smithi076.stdout:5765 punch from 0xaabc47b to 0xaacaab4, (0xe639 bytes) 2024-09-14T22:54:37.964 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:37.964 INFO:teuthology.orchestra.run.smithi076.stdout:5767 read 0x304a608 thru 0x3054065 (0x9a5e bytes) 2024-09-14T22:54:37.965 INFO:teuthology.orchestra.run.smithi076.stdout:5768 write 0xb975859 thru 0xb97df31 (0x86d9 bytes) 2024-09-14T22:54:37.968 INFO:teuthology.orchestra.run.smithi076.stdout:5769 write 0xfd6060 thru 0xfde9e3 (0x8984 bytes) 2024-09-14T22:54:37.969 INFO:teuthology.orchestra.run.smithi076.stdout:5770 write 0x78763fb thru 0x787d977 (0x757d bytes) 2024-09-14T22:54:37.972 INFO:teuthology.orchestra.run.smithi076.stdout:5775 write 0xe4de9a7 thru 0xe4e1bf7 (0x3251 bytes) 2024-09-14T22:54:37.975 INFO:teuthology.orchestra.run.smithi076.stdout:5776 read 0x61ac841 thru 0x61ade00 (0x15c0 bytes) 2024-09-14T22:54:37.979 INFO:teuthology.orchestra.run.smithi076.stdout:5777 trunc from 0xe588a4a to 0xba9966f 2024-09-14T22:54:37.990 INFO:teuthology.orchestra.run.smithi076.stdout:5782 writesame 0x8d5f72d thru 0x8d634f7 data_size 0x1(0x3dcb bytes) 2024-09-14T22:54:38.000 INFO:teuthology.orchestra.run.smithi076.stdout:5783 writesame 0x9f15e49 thru 0x9f1d750 data_size 0xc(0x7908 bytes) 2024-09-14T22:54:38.005 INFO:teuthology.orchestra.run.smithi076.stdout:5784 writesame 0x6d81ae2 thru 0x6d865a5 data_size 0x21(0x4ac4 bytes) 2024-09-14T22:54:38.007 INFO:teuthology.orchestra.run.smithi076.stdout:5785 clone 44 order 18 su 65536 sc 12 2024-09-14T22:54:38.009 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:38.007+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:38.056 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:38.056 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:38.065 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:38.109 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:38.108+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:38.167 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:38.176 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:38.196 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:38.196 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:54:38.196 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:54:38.196 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:54:38.240 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:38.239+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:38.300 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone43 from 0xba9966f (overlap 0x2c491e0) to 0x236db19 2024-09-14T22:54:38.311 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:38.309+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:38.332 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:38.333 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:38.342 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:38.397 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-14T22:54:38.400 INFO:teuthology.orchestra.run.smithi076.stdout:5786 trunc from 0xba9966f to 0xdb6170c 2024-09-14T22:54:38.409 INFO:teuthology.orchestra.run.smithi076.stdout:5787 write 0xd2feafa thru 0xd3036d7 (0x4bde bytes) 2024-09-14T22:54:38.411 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:38.410+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:38.413 INFO:teuthology.orchestra.run.smithi076.stdout:5788 write 0xad512ba thru 0xad5e413 (0xd15a bytes) 2024-09-14T22:54:38.417 INFO:teuthology.orchestra.run.smithi076.stdout:5789 read 0x4af829f thru 0x4afa249 (0x1fab bytes) 2024-09-14T22:54:38.417 INFO:teuthology.orchestra.run.smithi076.stdout:5790 write 0x1bfc68f thru 0x1bfdf13 (0x1885 bytes) 2024-09-14T22:54:38.420 INFO:teuthology.orchestra.run.smithi076.stdout:5791 write 0x2d09635 thru 0x2d0b000 (0x19cc bytes) 2024-09-14T22:54:38.424 INFO:teuthology.orchestra.run.smithi076.stdout:5792 punch from 0x3a8485c to 0x3a89c20, (0x53c4 bytes) 2024-09-14T22:54:38.424 INFO:teuthology.orchestra.run.smithi076.stdout:5793 read 0x172741a thru 0x1731aaf (0xa696 bytes) 2024-09-14T22:54:38.425 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:38.425 INFO:teuthology.orchestra.run.smithi076.stdout:5796 punch from 0xb20b73 to 0xb2d01b, (0xc4a8 bytes) 2024-09-14T22:54:38.425 INFO:teuthology.orchestra.run.smithi076.stdout:5798 punch from 0xa89ca09 to 0xa8ab986, (0xef7d bytes) 2024-09-14T22:54:38.431 INFO:teuthology.orchestra.run.smithi076.stdout:5799 write 0xe22039b thru 0xe22d87c (0xd4e2 bytes) 2024-09-14T22:54:38.434 INFO:teuthology.orchestra.run.smithi076.stdout:5800 clone 45 order 24 su 1048576 sc 6 2024-09-14T22:54:38.489 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:38.489 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:38.497 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 105, 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:54:38.226965+0000", 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:54:38.499 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5804032, 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "nodeep-scrub", 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:54:38.500 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:54:38.501 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:54:38.502 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:38.503 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:38.504 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.505 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:54:38.506 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:38.507 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:38.508 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:38.509 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:38.510 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:38.511 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:38.512 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 2.9900000095367432, 2024-09-14T22:54:38.513 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 2.9900000095367432, 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:54:38.514 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:54:38.515 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "105", 2024-09-14T22:54:38.516 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 46, 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 105, 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:38.517 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:38.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:38.519 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 2, 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 2, 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.520 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.5721282958984375, 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:38.521 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 94, 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:38.522 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:38.523 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:38.524 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:54:38.525 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.526 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.0684051513671875, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 58, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:38.528 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:38.529 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:38.530 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:54:38.531 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:38.532 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 94, 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:38.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:38.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:54:38.535 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:54:38.536 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:54:38.537 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:38.538 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 95, 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:54:38.539 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.540 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.541 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.542 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.543 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 86, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:38.544 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.546 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.548 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:54:38.549 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 95, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.550 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.551 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.552 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.553 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:54:38.554 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:38.555 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:38.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:38.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:38.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:38.560 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:38.561 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 75, 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:38.562 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:38.563 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:54:38.564 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:54:38.565 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.566 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.567 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:38.568 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:54:38.569 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:38.570 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:38.571 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.572 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "2.6", 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: 5, 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:54:38.573 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.2", 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: 4, 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: 5 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.7", 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: 7, 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:54:38.574 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.8", 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: 3, 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [], 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [], 2024-09-14T22:54:38.575 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [], 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:54:38.576 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:38.577 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:38.579 INFO:tasks.thrashosds.thrasher:j is {'epoch': 105, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:54:38.226965+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5721282958984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0684051513671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [5, 2]}, {'pgid': '3.0', 'osds': [6, 7]}, {'pgid': '3.2', 'osds': [4, 5]}, {'pgid': '3.7', 'osds': [7, 6]}, {'pgid': '3.8', 'osds': [3, 6]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:54:38.580 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:54:38.580 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:38.577+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:38.600 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:38.600 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:38.609 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:38.679 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:38.678+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:38.753 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:38.753 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:38.761 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:38.804 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:38.803+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:38.864 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:38.864 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:38.872 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:38.897 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:38.898 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:54:38.909 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-14T22:54:38.800041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1687614564,"num_objects":2600,"num_object_clones":493,"num_object_copies":5200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2600,"num_whiteouts":1563,"num_read":235155,"num_read_kb":1039333,"num_write":15250,"num_write_kb":555323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":77603230,"num_keys_recovered":54,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13733,"ondisk_log_size":13733,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1016488,"kb_used_data":697440,"kb_used_omap":12,"kb_used_meta":318707,"kb_avail":753958232,"statfs":{"total":773094113280,"available":772053229568,"internally_reserved":0,"allocated":714178560,"data_stored":827554821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":326356529},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":87873025,"num_objects":71,"num_object_clones":9,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":6,"num_read":5627,"num_read_kb":26630,"num_write":1359,"num_write_kb":13287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000551"},"pg_stats":[{"pgid":"3.1f","version":"101'552","reported_seq":7724,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.260427+0000","last_change":"2024-09-14T22:53:08.777340+0000","last_active":"2024-09-14T22:54:34.260427+0000","last_peered":"2024-09-14T22:54:34.260427+0000","last_clean":"2024-09-14T22:54:34.260427+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:54:34.260427+0000","last_undegraded":"2024-09-14T22:54:34.260427+0000","last_fullsized":"2024-09-14T22:54:34.260427+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:32:17.760623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001150756,"stat_sum":{"num_bytes":45787821,"num_objects":84,"num_object_clones":20,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":49,"num_read":8860,"num_read_kb":30640,"num_write":639,"num_write_kb":7457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'352","reported_seq":7544,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.327350+0000","last_change":"2024-09-14T22:54:28.173255+0000","last_active":"2024-09-14T22:54:34.327350+0000","last_peered":"2024-09-14T22:54:34.327350+0000","last_clean":"2024-09-14T22:54:34.327350+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:54:34.327350+0000","last_undegraded":"2024-09-14T22:54:34.327350+0000","last_fullsized":"2024-09-14T22:54:34.327350+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:59:34.744044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098261599999999996,"stat_sum":{"num_bytes":31467283,"num_objects":86,"num_object_clones":21,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":48,"num_read":7367,"num_read_kb":19186,"num_write":353,"num_write_kb":12428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'317","reported_seq":7928,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.272646+0000","last_change":"2024-09-14T22:53:08.770923+0000","last_active":"2024-09-14T22:54:34.272646+0000","last_peered":"2024-09-14T22:54:34.272646+0000","last_clean":"2024-09-14T22:54:34.272646+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:54:34.272646+0000","last_undegraded":"2024-09-14T22:54:34.272646+0000","last_fullsized":"2024-09-14T22:54:34.272646+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:03:47.704448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052998599999999998,"stat_sum":{"num_bytes":70952820,"num_objects":86,"num_object_clones":18,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":48,"num_read":8041,"num_read_kb":37457,"num_write":306,"num_write_kb":35231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"101'482","reported_seq":2562,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.287492+0000","last_change":"2024-09-14T22:53:08.771495+0000","last_active":"2024-09-14T22:54:34.287492+0000","last_peered":"2024-09-14T22:54:34.287492+0000","last_clean":"2024-09-14T22:54:34.287492+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:54:34.287492+0000","last_undegraded":"2024-09-14T22:54:34.287492+0000","last_fullsized":"2024-09-14T22:54:34.287492+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:10:17.002977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00113331,"stat_sum":{"num_bytes":53259702,"num_objects":65,"num_object_clones":10,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":38,"num_read":3674,"num_read_kb":11106,"num_write":548,"num_write_kb":8918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"101'240","reported_seq":4402,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.226687+0000","last_change":"2024-09-14T22:53:08.777584+0000","last_active":"2024-09-14T22:54:34.226687+0000","last_peered":"2024-09-14T22:54:34.226687+0000","last_clean":"2024-09-14T22:54:34.226687+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:54:34.226687+0000","last_undegraded":"2024-09-14T22:54:34.226687+0000","last_fullsized":"2024-09-14T22:54:34.226687+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:53:31.737734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013206229999999999,"stat_sum":{"num_bytes":92234551,"num_objects":73,"num_object_clones":11,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":43,"num_read":4135,"num_read_kb":53730,"num_write":249,"num_write_kb":18378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"101'428","reported_seq":12453,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.390183+0000","last_change":"2024-09-14T22:53:08.764073+0000","last_active":"2024-09-14T22:54:34.390183+0000","last_peered":"2024-09-14T22:54:34.390183+0000","last_clean":"2024-09-14T22:54:34.390183+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:54:34.390183+0000","last_undegraded":"2024-09-14T22:54:34.390183+0000","last_fullsized":"2024-09-14T22:54:34.390183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:24.449756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051139800000000004,"stat_sum":{"num_bytes":59227272,"num_objects":100,"num_object_clones":26,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":54,"num_read":13561,"num_read_kb":51111,"num_write":369,"num_write_kb":28892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'339","reported_seq":15300,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.307956+0000","last_change":"2024-09-14T22:53:51.943155+0000","last_active":"2024-09-14T22:54:34.307956+0000","last_peered":"2024-09-14T22:54:34.307956+0000","last_clean":"2024-09-14T22:54:34.307956+0000","last_became_active":"2024-09-14T22:53:51.942663+0000","last_became_peered":"2024-09-14T22:53:51.942663+0000","last_unstale":"2024-09-14T22:54:34.307956+0000","last_undegraded":"2024-09-14T22:54:34.307956+0000","last_fullsized":"2024-09-14T22:54:34.307956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:04:45.920598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031892299999999998,"stat_sum":{"num_bytes":75872617,"num_objects":93,"num_object_clones":16,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":58,"num_read":15419,"num_read_kb":91928,"num_write":319,"num_write_kb":37984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"101'582","reported_seq":1666,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.234563+0000","last_change":"2024-09-14T22:54:28.171666+0000","last_active":"2024-09-14T22:54:34.234563+0000","last_peered":"2024-09-14T22:54:34.234563+0000","last_clean":"2024-09-14T22:54:34.234563+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:54:34.234563+0000","last_undegraded":"2024-09-14T22:54:34.234563+0000","last_fullsized":"2024-09-14T22:54:34.234563+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:25.678725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062866099999999998,"stat_sum":{"num_bytes":33896235,"num_objects":75,"num_object_clones":9,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":51,"num_read":2618,"num_read_kb":22568,"num_write":798,"num_write_kb":10187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"101'528","reported_seq":6054,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.394061+0000","last_change":"2024-09-14T22:53:38.882316+0000","last_active":"2024-09-14T22:54:34.394061+0000","last_peered":"2024-09-14T22:54:34.394061+0000","last_clean":"2024-09-14T22:54:34.394061+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:54:34.394061+0000","last_undegraded":"2024-09-14T22:54:34.394061+0000","last_fullsized":"2024-09-14T22:54:34.394061+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:53:27.393766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011047710000000001,"stat_sum":{"num_bytes":35461576,"num_objects":66,"num_object_clones":8,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":34,"num_read":7349,"num_read_kb":33888,"num_write":736,"num_write_kb":27484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"101'305","reported_seq":8805,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.224081+0000","last_change":"2024-09-14T22:53:08.771175+0000","last_active":"2024-09-14T22:54:34.224081+0000","last_peered":"2024-09-14T22:54:34.224081+0000","last_clean":"2024-09-14T22:54:34.224081+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:54:34.224081+0000","last_undegraded":"2024-09-14T22:54:34.224081+0000","last_fullsized":"2024-09-14T22:54:34.224081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:50:33.806545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071186800000000005,"stat_sum":{"num_bytes":61286358,"num_objects":78,"num_object_clones":17,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":48,"num_read":9555,"num_read_kb":43942,"num_write":267,"num_write_kb":15829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"100'226","reported_seq":8673,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.216590+0000","last_change":"2024-09-14T22:53:08.829950+0000","last_active":"2024-09-14T22:54:34.216590+0000","last_peered":"2024-09-14T22:54:34.216590+0000","last_clean":"2024-09-14T22:54:34.216590+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:54:34.216590+0000","last_undegraded":"2024-09-14T22:54:34.216590+0000","last_fullsized":"2024-09-14T22:54:34.216590+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:29.536064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085951400000000002,"stat_sum":{"num_bytes":36029073,"num_objects":67,"num_object_clones":13,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":43,"num_read":9059,"num_read_kb":42623,"num_write":176,"num_write_kb":13380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"101'452","reported_seq":10482,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.323228+0000","last_change":"2024-09-14T22:53:08.763770+0000","last_active":"2024-09-14T22:54:34.323228+0000","last_peered":"2024-09-14T22:54:34.323228+0000","last_clean":"2024-09-14T22:54:34.323228+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:54:34.323228+0000","last_undegraded":"2024-09-14T22:54:34.323228+0000","last_fullsized":"2024-09-14T22:54:34.323228+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:28:45.863615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032182199999999998,"stat_sum":{"num_bytes":107682625,"num_objects":92,"num_object_clones":18,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":49,"num_read":11027,"num_read_kb":65963,"num_write":522,"num_write_kb":20906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":177,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.221237+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:54:34.221237+0000","last_peered":"2024-09-14T22:54:34.221237+0000","last_clean":"2024-09-14T22:54:34.221237+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:54:34.221237+0000","last_undegraded":"2024-09-14T22:54:34.221237+0000","last_fullsized":"2024-09-14T22:54:34.221237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"101'547","reported_seq":7264,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.240946+0000","last_change":"2024-09-14T22:53:08.776744+0000","last_active":"2024-09-14T22:54:34.240946+0000","last_peered":"2024-09-14T22:54:34.240946+0000","last_clean":"2024-09-14T22:54:34.240946+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:54:34.240946+0000","last_undegraded":"2024-09-14T22:54:34.240946+0000","last_fullsized":"2024-09-14T22:54:34.240946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:00:43.185109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060935299999999998,"stat_sum":{"num_bytes":103404698,"num_objects":75,"num_object_clones":14,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":42,"num_read":8830,"num_read_kb":41372,"num_write":651,"num_write_kb":24427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.221142+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:54:34.221142+0000","last_peered":"2024-09-14T22:54:34.221142+0000","last_clean":"2024-09-14T22:54:34.221142+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:54:34.221142+0000","last_undegraded":"2024-09-14T22:54:34.221142+0000","last_fullsized":"2024-09-14T22:54:34.221142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"101'250","reported_seq":1445,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.229003+0000","last_change":"2024-09-14T22:54:07.044085+0000","last_active":"2024-09-14T22:54:34.229003+0000","last_peered":"2024-09-14T22:54:34.229003+0000","last_clean":"2024-09-14T22:54:34.229003+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:54:34.229003+0000","last_undegraded":"2024-09-14T22:54:34.229003+0000","last_fullsized":"2024-09-14T22:54:34.229003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:39:57.834960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096004200000000004,"stat_sum":{"num_bytes":51899063,"num_objects":73,"num_object_clones":16,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":44,"num_read":1015,"num_read_kb":15061,"num_write":240,"num_write_kb":17347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.216676+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:54:34.216676+0000","last_peered":"2024-09-14T22:54:34.216676+0000","last_clean":"2024-09-14T22:54:34.216676+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:54:34.216676+0000","last_undegraded":"2024-09-14T22:54:34.216676+0000","last_fullsized":"2024-09-14T22:54:34.216676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"99'715","reported_seq":13129,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.220196+0000","last_change":"2024-09-14T22:53:51.942879+0000","last_active":"2024-09-14T22:54:34.220196+0000","last_peered":"2024-09-14T22:54:34.220196+0000","last_clean":"2024-09-14T22:54:34.220196+0000","last_became_active":"2024-09-14T22:53:51.942267+0000","last_became_peered":"2024-09-14T22:53:51.942267+0000","last_unstale":"2024-09-14T22:54:34.220196+0000","last_undegraded":"2024-09-14T22:54:34.220196+0000","last_fullsized":"2024-09-14T22:54:34.220196+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:33.097221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030890099999999999,"stat_sum":{"num_bytes":33598623,"num_objects":92,"num_object_clones":16,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":55,"num_read":15211,"num_read_kb":39688,"num_write":992,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":177,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.213009+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:54:34.213009+0000","last_peered":"2024-09-14T22:54:34.213009+0000","last_clean":"2024-09-14T22:54:34.213009+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:54:34.213009+0000","last_undegraded":"2024-09-14T22:54:34.213009+0000","last_fullsized":"2024-09-14T22:54:34.213009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"101'317","reported_seq":14604,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.221080+0000","last_change":"2024-09-14T22:54:28.178552+0000","last_active":"2024-09-14T22:54:34.221080+0000","last_peered":"2024-09-14T22:54:34.221080+0000","last_clean":"2024-09-14T22:54:34.221080+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:54:34.221080+0000","last_undegraded":"2024-09-14T22:54:34.221080+0000","last_fullsized":"2024-09-14T22:54:34.221080+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:53.906979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089494300000000004,"stat_sum":{"num_bytes":74985739,"num_objects":87,"num_object_clones":15,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":50,"num_read":13636,"num_read_kb":47951,"num_write":290,"num_write_kb":18867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"101'641","reported_seq":5507,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.228855+0000","last_change":"2024-09-14T22:53:08.763988+0000","last_active":"2024-09-14T22:54:34.228855+0000","last_peered":"2024-09-14T22:54:34.228855+0000","last_clean":"2024-09-14T22:54:34.228855+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:54:34.228855+0000","last_undegraded":"2024-09-14T22:54:34.228855+0000","last_fullsized":"2024-09-14T22:54:34.228855+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:59:25.408905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044722499999999999,"stat_sum":{"num_bytes":54964018,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":58,"num_read":7888,"num_read_kb":56596,"num_write":740,"num_write_kb":31470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":179,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.220939+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:54:34.220939+0000","last_peered":"2024-09-14T22:54:34.220939+0000","last_clean":"2024-09-14T22:54:34.220939+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:54:34.220939+0000","last_undegraded":"2024-09-14T22:54:34.220939+0000","last_fullsized":"2024-09-14T22:54:34.220939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"101'266","reported_seq":1087,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.269015+0000","last_change":"2024-09-14T22:53:08.829283+0000","last_active":"2024-09-14T22:54:34.269015+0000","last_peered":"2024-09-14T22:54:34.269015+0000","last_clean":"2024-09-14T22:54:34.269015+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:54:34.269015+0000","last_undegraded":"2024-09-14T22:54:34.269015+0000","last_fullsized":"2024-09-14T22:54:34.269015+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:00:27.780450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052083999999999995,"stat_sum":{"num_bytes":12773958,"num_objects":61,"num_object_clones":9,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":35,"num_read":1005,"num_read_kb":8326,"num_write":271,"num_write_kb":11527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":187,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.220232+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:54:34.220232+0000","last_peered":"2024-09-14T22:54:34.220232+0000","last_clean":"2024-09-14T22:54:34.220232+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:54:34.220232+0000","last_undegraded":"2024-09-14T22:54:34.220232+0000","last_fullsized":"2024-09-14T22:54:34.220232+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"101'514","reported_seq":8527,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.214053+0000","last_change":"2024-09-14T22:53:08.776528+0000","last_active":"2024-09-14T22:54:34.214053+0000","last_peered":"2024-09-14T22:54:34.214053+0000","last_clean":"2024-09-14T22:54:34.214053+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:54:34.214053+0000","last_undegraded":"2024-09-14T22:54:34.214053+0000","last_fullsized":"2024-09-14T22:54:34.214053+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:20:16.820568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450848,"stat_sum":{"num_bytes":27517432,"num_objects":79,"num_object_clones":14,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":46,"num_read":9669,"num_read_kb":21067,"num_write":648,"num_write_kb":11035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":177,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.216513+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:54:34.216513+0000","last_peered":"2024-09-14T22:54:34.216513+0000","last_clean":"2024-09-14T22:54:34.216513+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:54:34.216513+0000","last_undegraded":"2024-09-14T22:54:34.216513+0000","last_fullsized":"2024-09-14T22:54:34.216513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"101'273","reported_seq":5711,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.227049+0000","last_change":"2024-09-14T22:54:18.745625+0000","last_active":"2024-09-14T22:54:34.227049+0000","last_peered":"2024-09-14T22:54:34.227049+0000","last_clean":"2024-09-14T22:54:34.227049+0000","last_became_active":"2024-09-14T22:54:18.744069+0000","last_became_peered":"2024-09-14T22:54:18.744069+0000","last_unstale":"2024-09-14T22:54:34.227049+0000","last_undegraded":"2024-09-14T22:54:34.227049+0000","last_fullsized":"2024-09-14T22:54:34.227049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:31:27.664152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080380700000000002,"stat_sum":{"num_bytes":58509638,"num_objects":96,"num_object_clones":20,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":62,"num_read":5070,"num_read_kb":31214,"num_write":221,"num_write_kb":28665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":178,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.221032+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:54:34.221032+0000","last_peered":"2024-09-14T22:54:34.221032+0000","last_clean":"2024-09-14T22:54:34.221032+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:54:34.221032+0000","last_undegraded":"2024-09-14T22:54:34.221032+0000","last_fullsized":"2024-09-14T22:54:34.221032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":328,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.216570+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:54:34.216570+0000","last_peered":"2024-09-14T22:54:34.216570+0000","last_clean":"2024-09-14T22:54:34.216570+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:54:34.216570+0000","last_undegraded":"2024-09-14T22:54:34.216570+0000","last_fullsized":"2024-09-14T22:54:34.216570+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"101'486","reported_seq":8527,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.415399+0000","last_change":"2024-09-14T22:53:48.924907+0000","last_active":"2024-09-14T22:54:34.415399+0000","last_peered":"2024-09-14T22:54:34.415399+0000","last_clean":"2024-09-14T22:54:34.415399+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:54:34.415399+0000","last_undegraded":"2024-09-14T22:54:34.415399+0000","last_fullsized":"2024-09-14T22:54:34.415399+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:00:31.610923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065840099999999997,"stat_sum":{"num_bytes":49431084,"num_objects":96,"num_object_clones":21,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":57,"num_read":11029,"num_read_kb":33201,"num_write":442,"num_write_kb":12711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"101'384","reported_seq":1558,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.220630+0000","last_change":"2024-09-14T22:53:08.763829+0000","last_active":"2024-09-14T22:54:34.220630+0000","last_peered":"2024-09-14T22:54:34.220630+0000","last_clean":"2024-09-14T22:54:34.220630+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:54:34.220630+0000","last_undegraded":"2024-09-14T22:54:34.220630+0000","last_fullsized":"2024-09-14T22:54:34.220630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:17:52.374844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034751699999999999,"stat_sum":{"num_bytes":36504713,"num_objects":87,"num_object_clones":16,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":58,"num_read":2046,"num_read_kb":9011,"num_write":350,"num_write_kb":5961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"101'369","reported_seq":1471,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.308345+0000","last_change":"2024-09-14T22:53:08.829619+0000","last_active":"2024-09-14T22:54:34.308345+0000","last_peered":"2024-09-14T22:54:34.308345+0000","last_clean":"2024-09-14T22:54:34.308345+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:54:34.308345+0000","last_undegraded":"2024-09-14T22:54:34.308345+0000","last_fullsized":"2024-09-14T22:54:34.308345+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:14:08.652423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061471900000000001,"stat_sum":{"num_bytes":31221787,"num_objects":63,"num_object_clones":10,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":38,"num_read":2425,"num_read_kb":8059,"num_write":398,"num_write_kb":4441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"101'475","reported_seq":3067,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.234557+0000","last_change":"2024-09-14T22:53:08.761714+0000","last_active":"2024-09-14T22:54:34.234557+0000","last_peered":"2024-09-14T22:54:34.234557+0000","last_clean":"2024-09-14T22:54:34.234557+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:54:34.234557+0000","last_undegraded":"2024-09-14T22:54:34.234557+0000","last_fullsized":"2024-09-14T22:54:34.234557+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:16:01.060289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081304200000000004,"stat_sum":{"num_bytes":31681322,"num_objects":82,"num_object_clones":16,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":54,"num_read":4848,"num_read_kb":19608,"num_write":453,"num_write_kb":9105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"101'448","reported_seq":5032,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.221321+0000","last_change":"2024-09-14T22:54:28.177310+0000","last_active":"2024-09-14T22:54:34.221321+0000","last_peered":"2024-09-14T22:54:34.221321+0000","last_clean":"2024-09-14T22:54:34.221321+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:54:34.221321+0000","last_undegraded":"2024-09-14T22:54:34.221321+0000","last_fullsized":"2024-09-14T22:54:34.221321+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:43:09.322170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037933599999999997,"stat_sum":{"num_bytes":46091073,"num_objects":86,"num_object_clones":14,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":52,"num_read":5730,"num_read_kb":21623,"num_write":472,"num_write_kb":16955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"101'470","reported_seq":6452,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.410626+0000","last_change":"2024-09-14T22:53:08.830212+0000","last_active":"2024-09-14T22:54:34.410626+0000","last_peered":"2024-09-14T22:54:34.410626+0000","last_clean":"2024-09-14T22:54:34.410626+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:54:34.410626+0000","last_undegraded":"2024-09-14T22:54:34.410626+0000","last_fullsized":"2024-09-14T22:54:34.410626+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:03:03.198076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075108599999999998,"stat_sum":{"num_bytes":27687381,"num_objects":82,"num_object_clones":14,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":54,"num_read":7170,"num_read_kb":35464,"num_write":636,"num_write_kb":6477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"101'391","reported_seq":6403,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.252298+0000","last_change":"2024-09-14T22:53:08.829690+0000","last_active":"2024-09-14T22:54:34.252298+0000","last_peered":"2024-09-14T22:54:34.252298+0000","last_clean":"2024-09-14T22:54:34.252298+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:54:34.252298+0000","last_undegraded":"2024-09-14T22:54:34.252298+0000","last_fullsized":"2024-09-14T22:54:34.252298+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:03.371542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074121500000000004,"stat_sum":{"num_bytes":70922380,"num_objects":76,"num_object_clones":15,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":47,"num_read":7223,"num_read_kb":39767,"num_write":376,"num_write_kb":21305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"101'245","reported_seq":5924,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.320307+0000","last_change":"2024-09-14T22:53:08.829329+0000","last_active":"2024-09-14T22:54:34.320307+0000","last_peered":"2024-09-14T22:54:34.320307+0000","last_clean":"2024-09-14T22:54:34.320307+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:54:34.320307+0000","last_undegraded":"2024-09-14T22:54:34.320307+0000","last_fullsized":"2024-09-14T22:54:34.320307+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:24:42.365584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000379995,"stat_sum":{"num_bytes":28697658,"num_objects":80,"num_object_clones":17,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":48,"num_read":5658,"num_read_kb":21333,"num_write":218,"num_write_kb":11632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"101'359","reported_seq":8933,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.360802+0000","last_change":"2024-09-14T22:53:08.829747+0000","last_active":"2024-09-14T22:54:34.360802+0000","last_peered":"2024-09-14T22:54:34.360802+0000","last_clean":"2024-09-14T22:54:34.360802+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:54:34.360802+0000","last_undegraded":"2024-09-14T22:54:34.360802+0000","last_fullsized":"2024-09-14T22:54:34.360802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:19:58.870949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072248600000000005,"stat_sum":{"num_bytes":69295923,"num_objects":83,"num_object_clones":14,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":48,"num_read":10364,"num_read_kb":29034,"num_write":340,"num_write_kb":25575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"101'492","reported_seq":5054,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.234781+0000","last_change":"2024-09-14T22:53:08.829861+0000","last_active":"2024-09-14T22:54:34.234781+0000","last_peered":"2024-09-14T22:54:34.234781+0000","last_clean":"2024-09-14T22:54:34.234781+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:54:34.234781+0000","last_undegraded":"2024-09-14T22:54:34.234781+0000","last_fullsized":"2024-09-14T22:54:34.234781+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:46:13.943557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081970000000000003,"stat_sum":{"num_bytes":68662692,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":41,"num_read":5920,"num_read_kb":34373,"num_write":632,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"101'343","reported_seq":1169,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.230290+0000","last_change":"2024-09-14T22:53:38.883182+0000","last_active":"2024-09-14T22:54:34.230290+0000","last_peered":"2024-09-14T22:54:34.230290+0000","last_clean":"2024-09-14T22:54:34.230290+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:54:34.230290+0000","last_undegraded":"2024-09-14T22:54:34.230290+0000","last_fullsized":"2024-09-14T22:54:34.230290+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:53.747711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054753400000000002,"stat_sum":{"num_bytes":47449396,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":52,"num_read":1216,"num_read_kb":10505,"num_write":391,"num_write_kb":9582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"101'825","reported_seq":6028,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T22:54:34.266577+0000","last_change":"2024-09-14T22:53:08.829606+0000","last_active":"2024-09-14T22:54:34.266577+0000","last_peered":"2024-09-14T22:54:34.266577+0000","last_clean":"2024-09-14T22:54:34.266577+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:54:34.266577+0000","last_undegraded":"2024-09-14T22:54:34.266577+0000","last_fullsized":"2024-09-14T22:54:34.266577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:47:17.459445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000644452,"stat_sum":{"num_bytes":58567666,"num_objects":91,"num_object_clones":17,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":59,"num_read":8421,"num_read_kb":11838,"num_write":1046,"num_write_kb":5145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1687024177,"num_objects":2596,"num_object_clones":493,"num_object_copies":5192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2596,"num_whiteouts":1563,"num_read":235039,"num_read_kb":1039233,"num_write":15089,"num_write_kb":552905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":77603230,"num_keys_recovered":54,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":702578688,"data_stored":821051366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13613,"ondisk_log_size":13613,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121652,"kb_used_data":94628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94250188,"statfs":{"total":96636764160,"available":96512192512,"internally_reserved":0,"allocated":96899072,"data_stored":112435466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80920,"kb_used_data":45572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94290920,"statfs":{"total":96636764160,"available":96553902080,"internally_reserved":0,"allocated":46665728,"data_stored":54669862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124992,"kb_used_data":85488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94246848,"statfs":{"total":96636764160,"available":96508772352,"internally_reserved":0,"allocated":87539712,"data_stored":96828825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":216848,"kb_used_data":163452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94154992,"statfs":{"total":96636764160,"available":96414711808,"internally_reserved":0,"allocated":167374848,"data_stored":189707371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":130660,"kb_used_data":91068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94241180,"statfs":{"total":96636764160,"available":96502968320,"internally_reserved":0,"allocated":93253632,"data_stored":102487602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56820,"kb_used_data":21464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315020,"statfs":{"total":96636764160,"available":96578580480,"internally_reserved":0,"allocated":21979136,"data_stored":31753551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":120540,"kb_used_data":80944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94251300,"statfs":{"total":96636764160,"available":96513331200,"internally_reserved":0,"allocated":82886656,"data_stored":105994289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542170,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":164056,"kb_used_data":114824,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94207784,"statfs":{"total":96636764160,"available":96468770816,"internally_reserved":0,"allocated":117579776,"data_stored":133677855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114913280,"data_stored":131637604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81686528,"data_stored":105436338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20750336,"data_stored":31162851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":92114944,"data_stored":101976784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":166309888,"data_stored":189275277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":86339584,"data_stored":96270874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45469696,"data_stored":54111930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94994432,"data_stored":111179708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:54:38.911 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [4, 2, 0, 7] 2024-09-14T22:54:38.911 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.12', '4', '2', '0', '7'] 2024-09-14T22:54:38.911 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.12 4 2 0 7 2024-09-14T22:54:38.913 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:38.912+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:38.981 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:38.980+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:39.015 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:39.015 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:39.023 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:39.082 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:39.081+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:39.166 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:39.166 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:39.175 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:39.218 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:39.217+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:39.250 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:54:39.283 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:39.282+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:39.286 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:39.286 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:39.296 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:39.322 INFO:teuthology.orchestra.run.smithi076.stdout:truncating image image_client.0-clone44 from 0xe22d87d (overlap 0xba9966f) to 0x829e94d 2024-09-14T22:54:39.385 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:39.383+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:39.476 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:39.477 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:39.486 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:39.484+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:39.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:39.587 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:39.585+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:39.629 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-14T22:54:39.641 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:39.641 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:39.652 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:39.674 INFO:teuthology.orchestra.run.smithi076.stdout:5801 read 0xb622431 thru 0xb630df3 (0xe9c3 bytes) 2024-09-14T22:54:39.676 INFO:teuthology.orchestra.run.smithi076.stdout:5802 write 0x1cf124a thru 0x1cfc1f6 (0xafad bytes) 2024-09-14T22:54:39.688 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:39.686+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:39.755 INFO:teuthology.orchestra.run.smithi076.stdout:5803 writesame 0xa14ae3c thru 0xa153993 data_size 0x38(0x8b58 bytes) 2024-09-14T22:54:39.789 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:39.787+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:39.790 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:39.790 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:39.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:39.824 INFO:teuthology.orchestra.run.smithi076.stdout:5804 writesame 0x2426534 thru 0x242fd21 data_size 0x1(0x97ee bytes) 2024-09-14T22:54:39.890 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:39.889+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:39.901 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:39.913 INFO:teuthology.orchestra.run.smithi076.stdout:5805 punch from 0x81db7e9 to 0x81e7ae7, (0xc2fe bytes) 2024-09-14T22:54:39.913 INFO:teuthology.orchestra.run.smithi076.stdout:5806 write 0x85f869c thru 0x860251e (0x9e83 bytes) 2024-09-14T22:54:39.916 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:39.916 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:39.930 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:39.984 INFO:teuthology.orchestra.run.smithi076.stdout:5807 write 0x18c005c thru 0x18cc3bb (0xc360 bytes) 2024-09-14T22:54:39.986 INFO:teuthology.orchestra.run.smithi076.stdout:5808 write 0x1814ea3 thru 0x18173c6 (0x2524 bytes) 2024-09-14T22:54:39.989 INFO:teuthology.orchestra.run.smithi076.stdout:5809 read 0x1956076 thru 0x1963f30 (0xdebb bytes) 2024-09-14T22:54:39.989 INFO:teuthology.orchestra.run.smithi076.stdout:5810 write 0x9ba98a2 thru 0x9bb4e26 (0xb585 bytes) 2024-09-14T22:54:39.990 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:39.989+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:39.992 INFO:teuthology.orchestra.run.smithi076.stdout:5811 writesame 0x3da62eb thru 0x3dad2f9 data_size 0x1(0x700f bytes) 2024-09-14T22:54:40.037 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:40.038 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:40.055 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:40.055 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:40.071 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:40.071 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:40.086 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:40.086 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:40.086 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:40.086 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:40.086 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.086982140000003, 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.087 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.088 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:54:40.089 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.090 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.091 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.086963101999999, 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.092 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.093 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:54:40.094 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.095 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.096 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.086927146000001, 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.097 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.098 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.099 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.100 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:54:40.101 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.086906026000001, 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:54:40.102 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:40.103 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:54:40.104 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:54:40.105 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:54:40.106 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.107 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.085699112, 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.108 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:40.109 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.110 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.111 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.112 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.083869665000002, 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.113 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:54:40.114 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:40.115 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:54:40.116 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.083315378000002, 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.117 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:40.118 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:54:40.119 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:54:40.120 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.121 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.081330659999999, 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.122 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.123 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:54:40.124 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.125 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.080978340000001, 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.126 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:40.127 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.128 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.129 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.130 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.080694897000001, 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.131 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:54:40.132 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.133 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.134 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.080406799999999, 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.135 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.136 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:54:40.137 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.138 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.139 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.080031972, 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.140 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.141 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.142 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.143 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.144 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:40.145 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.079711156999998, 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.146 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:54:40.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:54:40.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.066025922000001, 2024-09-14T22:54:40.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:40.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:54:40.154 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.065691113, 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.155 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:54:40.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:54:40.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.065360951000002, 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.159 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:54:40.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.065140165000003, 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.164 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:40.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:40.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:54:40.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:54:40.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.064870864, 2024-09-14T22:54:40.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:54:40.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.064683768000002, 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.179 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:54:40.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.064365608999999, 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:54:40.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.187 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.188 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.188 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:40.189 INFO:teuthology.orchestra.run.smithi076.stdout:5812 read 0xc090085 thru 0xc097e9e (0x7e1a bytes) 2024-09-14T22:54:40.189 INFO:teuthology.orchestra.run.smithi076.stdout:5815 read 0x1e7540d thru 0x1e787df (0x33d3 bytes) 2024-09-14T22:54:40.189 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:40.189 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:40.190 INFO:teuthology.orchestra.run.smithi076.stdout:5816 read 0x64ac6f6 thru 0x64b2e7b (0x6786 bytes) 2024-09-14T22:54:40.190 INFO:teuthology.orchestra.run.smithi076.stdout:5817 read 0xbe8ac8 thru 0xbf3fe5 (0xb51e bytes) 2024-09-14T22:54:40.190 INFO:teuthology.orchestra.run.smithi076.stdout:5818 read 0x18e57c5 thru 0x18e61f6 (0xa32 bytes) 2024-09-14T22:54:40.190 INFO:teuthology.orchestra.run.smithi076.stdout:5822 write 0x11da0dd thru 0x11e4b89 (0xaaad bytes) 2024-09-14T22:54:40.191 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:40.192 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:40.192 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:40.192 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:40.192 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:40.193 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:40.193 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:40.193 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:40.194 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:40.194 INFO:teuthology.orchestra.run.smithi076.stdout:5824 writesame 0x15ba8f8 thru 0x15c0379 data_size 0x23(0x5a82 bytes) 2024-09-14T22:54:40.194 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:40.187+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:40.264 INFO:teuthology.orchestra.run.smithi076.stderr:set 3.12 pg_upmap_items mapping to [4->2,0->7] 2024-09-14T22:54:40.264 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:54:40.267 INFO:teuthology.orchestra.run.smithi076.stdout:5825 trunc from 0xe22d87d to 0x9318ddc 2024-09-14T22:54:40.290 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:40.288+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:40.345 INFO:teuthology.orchestra.run.smithi076.stdout:5826 write 0x5e88ebf thru 0x5e90625 (0x7767 bytes) 2024-09-14T22:54:40.347 INFO:teuthology.orchestra.run.smithi076.stdout:5827 write 0x51a19f1 thru 0x51b060d (0xec1d bytes) 2024-09-14T22:54:40.354 INFO:teuthology.orchestra.run.smithi076.stdout:5828 write 0x9e08db4 thru 0x9e10316 (0x7563 bytes) 2024-09-14T22:54:40.356 INFO:teuthology.orchestra.run.smithi076.stdout:5829 write 0x9c3eab3 thru 0x9c3fb6a (0x10b8 bytes) 2024-09-14T22:54:40.357 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.363 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:40.363 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:40.363 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:40.364 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:40.365 INFO:teuthology.orchestra.run.smithi076.stdout:5831 writesame 0xc9bc58a thru 0xc9bd956 data_size 0x1(0x13cd bytes) 2024-09-14T22:54:40.365 INFO:teuthology.orchestra.run.smithi076.stdout:5832 read 0x4616945 thru 0x461c74e (0x5e0a bytes) 2024-09-14T22:54:40.373 INFO:teuthology.orchestra.run.smithi076.stdout:5834 trunc from 0xc9bd957 to 0x7c057c1 2024-09-14T22:54:40.376 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:40.380 INFO:teuthology.orchestra.run.smithi076.stdout:5836 read 0x2913640 thru 0x291b08f (0x7a50 bytes) 2024-09-14T22:54:40.386 INFO:teuthology.orchestra.run.smithi076.stdout:5838 write 0x85dd6d8 thru 0x85e1134 (0x3a5d bytes) 2024-09-14T22:54:40.388 INFO:teuthology.orchestra.run.smithi076.stdout:5840 punch from 0x720557f to 0x720a2c6, (0x4d47 bytes) 2024-09-14T22:54:40.388 INFO:teuthology.orchestra.run.smithi076.stdout:5841 punch from 0xe3d0d9 to 0xe4b9cc, (0xe8f3 bytes) 2024-09-14T22:54:40.388 INFO:teuthology.orchestra.run.smithi076.stdout:5842 read 0x7cd62ba thru 0x7ce3bd5 (0xd91c bytes) 2024-09-14T22:54:40.389 INFO:teuthology.orchestra.run.smithi076.stdout:5843 punch from 0x29d4925 to 0x29da788, (0x5e63 bytes) 2024-09-14T22:54:40.391 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:40.389+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:40.394 INFO:teuthology.orchestra.run.smithi076.stdout:5845 write 0x96b2d18 thru 0x96bd68c (0xa975 bytes) 2024-09-14T22:54:40.396 INFO:teuthology.orchestra.run.smithi076.stdout:5846 punch from 0x283fb0a to 0x2841f57, (0x244d bytes) 2024-09-14T22:54:40.397 INFO:teuthology.orchestra.run.smithi076.stdout:5847 writesame 0x35fcf68 thru 0x35ff1d0 data_size 0x1(0x2269 bytes) 2024-09-14T22:54:40.402 INFO:teuthology.orchestra.run.smithi076.stdout:5848 read 0x5a2bb3 thru 0x5a317f (0x5cd bytes) 2024-09-14T22:54:40.403 INFO:teuthology.orchestra.run.smithi076.stdout:5852 punch from 0x4baa9fd to 0x4bb6abd, (0xc0c0 bytes) 2024-09-14T22:54:40.403 INFO:teuthology.orchestra.run.smithi076.stdout:5853 write 0x7197aa4 thru 0x719a6b7 (0x2c14 bytes) 2024-09-14T22:54:40.405 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.405 INFO:teuthology.orchestra.run.smithi076.stdout:5858 read 0x2d1c89a thru 0x2d1fbaa (0x3311 bytes) 2024-09-14T22:54:40.414 INFO:teuthology.orchestra.run.smithi076.stdout:5860 writesame 0xcee0a35 thru 0xceeb3f9 data_size 0x21(0xa9c5 bytes) 2024-09-14T22:54:40.414 INFO:teuthology.orchestra.run.smithi076.stdout:5861 read 0xbfd3dd thru 0xbffc3c (0x2860 bytes) 2024-09-14T22:54:40.425 INFO:teuthology.orchestra.run.smithi076.stdout:5862 trunc from 0xceeb3fa to 0x46fa40c 2024-09-14T22:54:40.492 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:40.491+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:40.494 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:40.494 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:40.494 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:40.494 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:40.494 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:40.500 INFO:teuthology.orchestra.run.smithi076.stdout:5865 write 0x229c1d8 thru 0x22a1250 (0x5079 bytes) 2024-09-14T22:54:40.502 INFO:teuthology.orchestra.run.smithi076.stdout:5866 read 0x40e20cd thru 0x40eb63a (0x956e bytes) 2024-09-14T22:54:40.503 INFO:teuthology.orchestra.run.smithi076.stdout:5867 read 0x178f5db thru 0x178f7b3 (0x1d9 bytes) 2024-09-14T22:54:40.503 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:40.504 INFO:teuthology.orchestra.run.smithi076.stdout:5868 read 0x3f19e13 thru 0x3f25020 (0xb20e bytes) 2024-09-14T22:54:40.505 INFO:teuthology.orchestra.run.smithi076.stdout:5869 writesame 0x3b2e2d thru 0x3ba515 data_size 0x49(0x76e9 bytes) 2024-09-14T22:54:40.513 INFO:teuthology.orchestra.run.smithi076.stdout:5870 write 0x5de253f thru 0x5deacec (0x87ae bytes) 2024-09-14T22:54:40.520 INFO:teuthology.orchestra.run.smithi076.stdout:5871 write 0x6488061 thru 0x648ce0e (0x4dae bytes) 2024-09-14T22:54:40.522 INFO:teuthology.orchestra.run.smithi076.stdout:5873 write 0x25f4b14 thru 0x25ff7f8 (0xace5 bytes) 2024-09-14T22:54:40.538 INFO:teuthology.orchestra.run.smithi076.stdout:5874 write 0xdb243e9 thru 0xdb2ebfe (0xa816 bytes) 2024-09-14T22:54:40.543 INFO:teuthology.orchestra.run.smithi076.stdout:5876 writesame 0xbdc0dee thru 0xbdc415f data_size 0xf(0x3372 bytes) 2024-09-14T22:54:40.547 INFO:teuthology.orchestra.run.smithi076.stdout:5877 punch from 0x8bdb745 to 0x8be2731, (0x6fec bytes) 2024-09-14T22:54:40.547 INFO:teuthology.orchestra.run.smithi076.stdout:5880 read 0x8edb361 thru 0x8ee4cb8 (0x9958 bytes) 2024-09-14T22:54:40.548 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.548 INFO:teuthology.orchestra.run.smithi076.stdout:5883 write 0xb951482 thru 0xb95eb32 (0xd6b1 bytes) 2024-09-14T22:54:40.551 INFO:teuthology.orchestra.run.smithi076.stdout:5884 read 0x430940b thru 0x4310e97 (0x7a8d bytes) 2024-09-14T22:54:40.551 INFO:teuthology.orchestra.run.smithi076.stdout:5885 read 0x21e910d thru 0x21f8ac7 (0xf9bb bytes) 2024-09-14T22:54:40.552 INFO:teuthology.orchestra.run.smithi076.stdout:5886 write 0x8bc141f thru 0x8bcd0ef (0xbcd1 bytes) 2024-09-14T22:54:40.569 INFO:teuthology.orchestra.run.smithi076.stdout:5887 trunc from 0xdb2ebff to 0x1ec5f56 2024-09-14T22:54:40.585 INFO:teuthology.orchestra.run.smithi076.stdout:5888 read 0x1da03c7 thru 0x1daffc1 (0xfbfb bytes) 2024-09-14T22:54:40.591 INFO:teuthology.orchestra.run.smithi076.stdout:5889 write 0x2d043da thru 0x2d0b073 (0x6c9a bytes) 2024-09-14T22:54:40.592 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:40.591+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:40.593 INFO:teuthology.orchestra.run.smithi076.stdout:5890 read 0x5b0d7b thru 0x5b6e2e (0x60b4 bytes) 2024-09-14T22:54:40.594 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.595 INFO:teuthology.orchestra.run.smithi076.stdout:5894 trunc from 0x2d0b074 to 0x1fca5d4 2024-09-14T22:54:40.612 INFO:teuthology.orchestra.run.smithi076.stdout:5896 write 0x6be95a8 thru 0x6bf90e8 (0xfb41 bytes) 2024-09-14T22:54:40.615 INFO:teuthology.orchestra.run.smithi076.stdout:5897 read 0x2aefdbc thru 0x2af0649 (0x88e bytes) 2024-09-14T22:54:40.616 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.616 INFO:teuthology.orchestra.run.smithi076.stdout:5899 read 0x5ff9ff5 thru 0x60064b8 (0xc4c4 bytes) 2024-09-14T22:54:40.624 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 73.413000976000006, 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:54:40.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:54:40.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.629 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.289339202000001, 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:54:40.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:54:40.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.357571731, 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.635 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:54:40.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:54:40.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:54:40.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 26.447625489, 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:54:40.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:54:40.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.643 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172324 3.4 3:213525dd:::rbd_data.113598f99d1a.00000000000000b5:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 26.434835479, 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051499340000000001, 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.644 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172324 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186688+0000", 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1829999999999999e-06 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186693+0000", 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4100000000000001e-06 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186705+0000", 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2169e-05 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186712+0000", 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9789999999999996e-06 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186758+0000", 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6687000000000002e-05 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186839+0000", 2024-09-14T22:54:40.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-05 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186980+0000", 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000141569 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191467+0000", 2024-09-14T22:54:40.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0044866840000000003 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191734+0000", 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026690299999999998 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191793+0000", 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9447000000000002e-05 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191837+0000", 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3433000000000002e-05 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:54:40.651 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 17.498964846, 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.652 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:54:40.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.657 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.042887969000001, 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.658 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.662 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.042860646999999, 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.663 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:54:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:54:40.666 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.042449156, 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:54:40.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.040789888999999, 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:54:40.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:54:40.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:54:40.674 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.036758465, 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:40.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:54:40.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194089 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.036170799000001, 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005182201, 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.680 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194089 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.681 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585353+0000", 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8899999999999998e-07 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585357+0000", 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8470000000000003e-06 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585369+0000", 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1992e-05 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585377+0000", 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6090000000000003e-06 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590237+0000", 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048596910000000002 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590352+0000", 2024-09-14T22:54:40.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000115387 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590534+0000", 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018168599999999999 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.685 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194119 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.034935439, 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051579030000000001, 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.686 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194119 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:40.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8200000000000001e-07 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586590+0000", 2024-09-14T22:54:40.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586596+0000", 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.212e-06 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586601+0000", 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8289999999999998e-06 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591330+0000", 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047292439999999996 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591418+0000", 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8448000000000004e-05 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591745+0000", 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00032651799999999998 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:40.691 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.034914989000001, 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:40.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:54:40.695 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.034895315, 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.696 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:40.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.700 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.034856660999999, 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:54:40.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:54:40.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.034836780999999, 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.705 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:40.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:54:40.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:40.709 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.661665588, 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:54:40.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:40.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:54:40.715 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 9.660664294, 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:54:40.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:40.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.3546081030000003, 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:54:40.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:40.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:54:40.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:54:40.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:54:40.728 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:40.729 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:40.729 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:40.730 INFO:teuthology.orchestra.run.smithi076.stdout:5900 write 0xc7bc8c7 thru 0xc7c88b6 (0xbff0 bytes) 2024-09-14T22:54:40.730 INFO:teuthology.orchestra.run.smithi076.stdout:5901 write 0x4a66b9a thru 0x4a6700f (0x476 bytes) 2024-09-14T22:54:40.730 INFO:teuthology.orchestra.run.smithi076.stdout:5902 write 0x183a234 thru 0x183c91f (0x26ec bytes) 2024-09-14T22:54:40.731 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5903 read 0xa8fae13 thru 0xa8fbd98 (0xf86 bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5904 write 0x6bfe6a4 thru 0x6bfff6b (0x18c8 bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5905 writesame 0x1fac705 thru 0x1fad4e3 data_size 0x1(0xddf bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5907 read 0x11982e1 thru 0x119f500 (0x7220 bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5910 write 0x6550715 thru 0x6554f63 (0x484f bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5911 read 0x56ad7da thru 0x56b8cb2 (0xb4d9 bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5913 read 0x834822e thru 0x834c208 (0x3fdb bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5914 writesame 0xb8ab7a8 thru 0xb8ba854 data_size 0x1(0xf0ad bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5916 writesame 0xbc2e712 thru 0xbc3858f data_size 0x1(0x9e7e bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:5917 read 0x4169289 thru 0x4171c5b (0x89d3 bytes) 2024-09-14T22:54:40.734 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5919 punch from 0x476653 to 0x47b4d5, (0x4e82 bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5920 flatten 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5923 writesame 0xd8526cf thru 0xd85eea8 data_size 0x3(0xc7da bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5924 punch from 0xe821d5 to 0xe8a9e7, (0x8812 bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5926 read 0x344c776 thru 0x344f21e (0x2aa9 bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5928 read 0x844fcd6 thru 0x844fd2e (0x59 bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5929 read 0x5b5a764 thru 0x5b66d0a (0xc5a7 bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5930 write 0xbbe0e5a thru 0xbbec0b9 (0xb260 bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5931 punch from 0xaeadc06 to 0xaebb4a0, (0xd89a bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5933 write 0x6ed1e5b thru 0x6ed382b (0x19d1 bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5934 read 0x4a88aa2 thru 0x4a9771b (0xec7a bytes) 2024-09-14T22:54:40.735 INFO:teuthology.orchestra.run.smithi076.stdout:5935 punch from 0xc0cdf38 to 0xc0ce8c3, (0x98b bytes) 2024-09-14T22:54:40.736 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:40.729+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:40.737 INFO:teuthology.orchestra.run.smithi076.stdout:5936 trunc from 0xd85eea9 to 0x467312 2024-09-14T22:54:40.774 INFO:teuthology.orchestra.run.smithi076.stdout:5938 writesame 0x9bac358 thru 0x9bae947 data_size 0x4(0x25f0 bytes) 2024-09-14T22:54:40.780 INFO:teuthology.orchestra.run.smithi076.stdout:5939 trunc from 0x9bae948 to 0xc19b2a2 2024-09-14T22:54:40.783 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.783 INFO:teuthology.orchestra.run.smithi076.stdout:5943 write 0x3773a19 thru 0x37792ef (0x58d7 bytes) 2024-09-14T22:54:40.784 INFO:teuthology.orchestra.run.smithi076.stdout:5944 write 0x45e82dd thru 0x45ef2f0 (0x7014 bytes) 2024-09-14T22:54:40.786 INFO:teuthology.orchestra.run.smithi076.stdout:5945 read 0x7e6ce75 thru 0x7e740c9 (0x7255 bytes) 2024-09-14T22:54:40.786 INFO:teuthology.orchestra.run.smithi076.stdout:5946 read 0x19789ba thru 0x1978a41 (0x88 bytes) 2024-09-14T22:54:40.799 INFO:teuthology.orchestra.run.smithi076.stdout:5947 trunc from 0xc19b2a2 to 0xcd7ad6 2024-09-14T22:54:40.822 INFO:teuthology.orchestra.run.smithi076.stdout:5948 trunc from 0xcd7ad6 to 0xd9b6d6a 2024-09-14T22:54:40.827 INFO:teuthology.orchestra.run.smithi076.stdout:5953 read 0x3295b2e thru 0x3296d25 (0x11f8 bytes) 2024-09-14T22:54:40.828 INFO:teuthology.orchestra.run.smithi076.stdout:5954 write 0x628264f thru 0x6291600 (0xefb2 bytes) 2024-09-14T22:54:40.831 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:40.830+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:40.833 INFO:teuthology.orchestra.run.smithi076.stdout:5955 trunc from 0xd9b6d6a to 0xc529175 2024-09-14T22:54:40.838 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.838 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.838 INFO:teuthology.orchestra.run.smithi076.stdout:5958 read 0xb77f58a thru 0xb78685a (0x72d1 bytes) 2024-09-14T22:54:40.838 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.844 INFO:teuthology.orchestra.run.smithi076.stdout:5960 write 0xd89e9e2 thru 0xd8a6b86 (0x81a5 bytes) 2024-09-14T22:54:40.847 INFO:teuthology.orchestra.run.smithi076.stdout:5961 writesame 0x9d82c86 thru 0x9d92b43 data_size 0x9(0xfebe bytes) 2024-09-14T22:54:40.853 INFO:teuthology.orchestra.run.smithi076.stdout:5962 read 0x4a2b0ee thru 0x4a30b3c (0x5a4f bytes) 2024-09-14T22:54:40.855 INFO:teuthology.orchestra.run.smithi076.stdout:5963 trunc from 0xd8a6b87 to 0xbce0bc3 2024-09-14T22:54:40.859 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:40.859 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:40.859 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:40.859 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:40.866 INFO:teuthology.orchestra.run.smithi076.stdout:5964 writesame 0x1a06d7f thru 0x1a0fb4a data_size 0x42(0x8dcc bytes) 2024-09-14T22:54:40.869 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:40.869 INFO:teuthology.orchestra.run.smithi076.stdout:5966 writesame 0xa1061ea thru 0xa10e34f data_size 0x3(0x8166 bytes) 2024-09-14T22:54:40.887 INFO:teuthology.orchestra.run.smithi076.stdout:5967 trunc from 0xbce0bc3 to 0x2a2ff88 2024-09-14T22:54:40.901 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:40.901 INFO:teuthology.orchestra.run.smithi076.stdout:5969 read 0x2471960 thru 0x247ff72 (0xe613 bytes) 2024-09-14T22:54:40.911 INFO:teuthology.orchestra.run.smithi076.stdout:5970 write 0x60e054d thru 0x60e17c3 (0x1277 bytes) 2024-09-14T22:54:40.914 INFO:teuthology.orchestra.run.smithi076.stdout:5972 write 0x3810a62 thru 0x3817127 (0x66c6 bytes) 2024-09-14T22:54:40.931 INFO:teuthology.orchestra.run.smithi076.stdout:5973 writesame 0xe6d832e thru 0xe6e077a data_size 0x1(0x844d bytes) 2024-09-14T22:54:40.931 INFO:teuthology.orchestra.run.smithi076.stdout:5975 write 0x32d6196 thru 0x32e408a (0xdef5 bytes) 2024-09-14T22:54:40.932 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:40.930+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:40.932 INFO:teuthology.orchestra.run.smithi076.stdout:5976 write 0x81a421c thru 0x81a45de (0x3c3 bytes) 2024-09-14T22:54:40.935 INFO:teuthology.orchestra.run.smithi076.stdout:5978 read 0x2db4c9 thru 0x2e951d (0xe055 bytes) 2024-09-14T22:54:40.936 INFO:teuthology.orchestra.run.smithi076.stdout:5979 punch from 0xb415a6d to 0xb418d2b, (0x32be bytes) 2024-09-14T22:54:40.936 INFO:teuthology.orchestra.run.smithi076.stdout:5980 read 0xc50e3b0 thru 0xc51338c (0x4fdd bytes) 2024-09-14T22:54:40.937 INFO:teuthology.orchestra.run.smithi076.stdout:5981 writesame 0xcdc4c thru 0xd45d1 data_size 0xd(0x6986 bytes) 2024-09-14T22:54:40.939 INFO:teuthology.orchestra.run.smithi076.stdout:5982 write 0x418e301 thru 0x418ea42 (0x742 bytes) 2024-09-14T22:54:40.941 INFO:teuthology.orchestra.run.smithi076.stdout:5984 punch from 0x77a2758 to 0x77a6765, (0x400d bytes) 2024-09-14T22:54:40.960 INFO:teuthology.orchestra.run.smithi076.stdout:5985 trunc from 0xe6e077b to 0x12e33e7 2024-09-14T22:54:40.973 INFO:teuthology.orchestra.run.smithi076.stdout:5988 read 0x342c43 thru 0x3436bc (0xa7a bytes) 2024-09-14T22:54:40.974 INFO:teuthology.orchestra.run.smithi076.stdout:5989 read 0x1267949 thru 0x1276c08 (0xf2c0 bytes) 2024-09-14T22:54:40.985 INFO:teuthology.orchestra.run.smithi076.stdout:5991 writesame 0x49b3794 thru 0x49baa7e data_size 0x1f(0x72eb bytes) 2024-09-14T22:54:40.991 INFO:teuthology.orchestra.run.smithi076.stdout:5993 trunc from 0x49baa7f to 0x8d9e298 2024-09-14T22:54:40.997 INFO:teuthology.orchestra.run.smithi076.stdout:5995 punch from 0x887fe12 to 0x8880d07, (0xef5 bytes) 2024-09-14T22:54:40.998 INFO:teuthology.orchestra.run.smithi076.stdout:5996 punch from 0x4f2a72a to 0x4f2c834, (0x210a bytes) 2024-09-14T22:54:41.005 INFO:teuthology.orchestra.run.smithi076.stdout:5997 write 0xa659f8a thru 0xa667c65 (0xdcdc bytes) 2024-09-14T22:54:41.008 INFO:teuthology.orchestra.run.smithi076.stdout:5998 writesame 0x37df99e thru 0x37e069a data_size 0x19(0xcfd bytes) 2024-09-14T22:54:41.010 INFO:teuthology.orchestra.run.smithi076.stdout:5999 write 0x78c97ca thru 0x78d4b2d (0xb364 bytes) 2024-09-14T22:54:41.013 INFO:teuthology.orchestra.run.smithi076.stdout:compare and write disabled 2024-09-14T22:54:41.015 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:41.015 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:41.015 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:41.015 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:41.015 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:41.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:41.046 INFO:teuthology.orchestra.run.smithi076.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-14T22:54:41.047 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:41.045+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:41.133 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:41.132+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.279105031, 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:54:41.186 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.190 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.279084495000006, 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.191 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:41.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.195 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.279054496000001, 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.279023343000006, 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:54:41.200 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:54:41.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.279000687000007, 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:54:41.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:54:41.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278980185999998, 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.209 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.213 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278960424999994, 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:54:41.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278939640000004, 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.218 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:41.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:54:41.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.222 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278907227999994, 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:54:41.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278885829000004, 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.227 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:41.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.231 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.27882443, 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.232 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:54:41.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278784517000005, 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:54:41.236 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:54:41.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.240 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278754761000002, 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.241 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:54:41.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.245 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278724526000005, 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:54:41.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:54:41.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278699786999994, 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.250 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.254 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278676356999995, 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:54:41.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:41.259 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278580621000003, 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.263 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278551699000005, 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.264 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:54:41.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:41.268 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.278528374000004, 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:41.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:54:41.272 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.145830631000003, 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:41.273 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:41.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:41.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:54:41.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:54:41.278 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.278 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.278 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.278 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:41.278 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:41.278 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:41.279 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:41.280 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:41.277+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:41.380 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:41.379+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:41.390 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:41.391 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:41.391 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:41.391 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:41.400 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:41.482 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:41.481+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:41.502 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:41.503 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:41.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:41.582 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:41.581+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:41.614 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:41.614 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.558286914999996, 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.615 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.619 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.558091822999998, 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:54:41.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:54:41.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.557939032999997, 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.624 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:54:41.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:54:41.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.557734010999994, 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:54:41.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:54:41.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.557463345000002, 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:54:41.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:54:41.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:54:41.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:54:41.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.557201184999997, 2024-09-14T22:54:41.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:54:41.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:54:41.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.557042332999998, 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:54:41.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:41.647 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.556855537000004, 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:41.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:54:41.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:41.652 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.556653291999993, 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:54:41.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:54:41.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.657 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.556444933999998, 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:54:41.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:54:41.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:54:41.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.556131217000001, 2024-09-14T22:54:41.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:54:41.664 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.664 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.664 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:54:41.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.665 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:41.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:54:41.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:54:41.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:54:41.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:54:41.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:54:41.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:54:41.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:54:41.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:54:41.670 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.670 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.670 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.555925454999993, 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.671 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:41.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:54:41.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:54:41.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:54:41.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:54:41.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:54:41.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.555756579000004, 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:54:41.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.555485884999996, 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:54:41.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.686 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.555336609999998, 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:54:41.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.555095367999996, 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:54:41.691 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:54:41.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:54:41.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:54:41.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:41.697 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.246405335999995, 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:41.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:54:41.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:54:41.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:54:41.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:54:41.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153281 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'291, mlcod=72'289)", 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.176895874000003, 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060467871999999999, 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.704 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.705 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:41.705 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:41.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435411+0000", 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5589999999999999e-06 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435419+0000", 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.909e-06 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435423+0000", 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5140000000000003e-06 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435458+0000", 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965000000000002e-05 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435470+0000", 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1559999999999999e-05 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495827+0000", 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060356819999999999 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495874+0000", 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7003999999999999e-05 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153282 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'292, mlcod=72'289)", 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.176264158000002, 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060246112999999997, 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.709 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436038+0000", 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436042+0000", 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8920000000000002e-06 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495165+0000", 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059122945000000003 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495179+0000", 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4185000000000001e-05 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495346+0000", 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016677400000000001 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495373+0000", 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7172e-05 2024-09-14T22:54:41.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496245+0000", 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00087206100000000004 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496284+0000", 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8525000000000001e-05 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:187784 3.2 3:45bacf70:::rbd_data.11354c7c4fd6.0000000000000028:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.781103437999999, 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.91484284000000005, 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 187784 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.714 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831199+0000", 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2200000000000002e-07 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831203+0000", 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1019999999999999e-06 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831213+0000", 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0087e-05 2024-09-14T22:54:41.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831218+0000", 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2299999999999999e-06 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831275+0000", 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6200999999999999e-05 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:41.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831286+0000", 2024-09-14T22:54:41.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0838e-05 2024-09-14T22:54:41.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:41.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.744712+0000", 2024-09-14T22:54:41.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.91342615800000004 2024-09-14T22:54:41.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:41.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.744890+0000", 2024-09-14T22:54:41.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000178488 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 5", 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745032+0000", 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014174500000000001 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:41.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745748+0000", 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00071614400000000005 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745984+0000", 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023544500000000001 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.746015+0000", 2024-09-14T22:54:41.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1683000000000001e-05 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.746041+0000", 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5797000000000001e-05 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:41.722 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:41.724 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:41.724 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:41.723+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:41.825 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:41.823+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:41.828 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:41.828 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:41.828 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:41.828 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:41.837 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:41.926 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:41.923+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:41.939 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:41.939 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:41.940 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:41.940 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:41.940 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:41.949 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:42.033 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:42.031+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:42.094 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:42.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.196196168, 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.095 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:54:42.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:54:42.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:54:42.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.570402048999995, 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.570314057000004, 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.106 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.110 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.570080140000002, 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.114 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.569899245000002, 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:54:42.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.569599741999994, 2024-09-14T22:54:42.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:42.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.123 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.569253532000005, 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.124 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:54:42.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.128 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.568889803999994, 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:42.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.132 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.568300187000006, 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:54:42.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:54:42.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.568097949999995, 2024-09-14T22:54:42.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:54:42.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.567908138999996, 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.142 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.146 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.567596136000006, 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:54:42.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.567353648999998, 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:54:42.151 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:54:42.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:54:42.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:54:42.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:54:42.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:54:42.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:54:42.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:54:42.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:54:42.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:54:42.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:54:42.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:54:42.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:54:42.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:54:42.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:54:42.159 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.159 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.159 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.159 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.160 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:42.160 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.567194686999997, 2024-09-14T22:54:42.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:54:42.160 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.160 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.160 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.161 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:54:42.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.162 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:42.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:42.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:54:42.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:54:42.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:54:42.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:54:42.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:54:42.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:54:42.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:54:42.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:54:42.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:54:42.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:54:42.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:54:42.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:54:42.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:54:42.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:54:42.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.170 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.170 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:42.170 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.566560417000005, 2024-09-14T22:54:42.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:54:42.170 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.171 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.171 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.171 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.171 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.171 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.171 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.566346307000003, 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.176 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:54:42.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:54:42.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.566209837000002, 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:54:42.180 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:54:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:54:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.565957357000002, 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:54:42.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:42.189 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.467059033, 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:54:42.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:54:42.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.971572404, 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:54:42.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:54:42.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:42.203 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:42.204 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:42.204 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:42.205 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:42.204+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:42.206 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:42.287 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:54:42.331 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:42.330+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:42.343 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:42.362 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:42.364 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:42.364 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:42.364 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:42.364 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:42.373 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:42.379 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:42.396 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:42.396 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:42.406 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:42.405+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:42.417 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:42.417 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:42.433 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:42.434 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:42.450 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:42.450 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:42.466 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:42.466 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:42.480 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:42.480 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:42.489 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:42.533 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:42.531+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:42.593 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:42.593 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:42.601 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:42.608 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:42.608+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:42.710 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:42.707+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:42.745 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:42.745 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:42.754 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:42.838 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:42.836+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:42.897 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:42.897 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:42.906 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:42.911 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:42.910+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:43.012 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:43.010+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:43.051 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:43.051 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:43.060 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:43.113 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:43.112+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:43.166 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:43.167 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:43.175 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:43.216 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:43.215+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:43.271 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:54:43.315 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:43.313+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:43.329 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:43.329 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:43.338 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:43.422 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:43.421+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:43.481 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:43.482 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:43.490 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:43.531 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:43.530+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:43.617 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:43.615+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:43.634 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:43.634 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:43.644 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:43.719 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:43.717+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:43.746 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:43.746 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:43.755 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:43.819 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:43.818+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:43.857 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:43.858 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:43.866 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:43.921 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:43.920+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:43.969 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:43.970 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:43.978 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:44.062 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:44.061+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:44.121 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:44.121 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:44.122 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:44.121+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:44.129 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:44.223 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:44.221+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:44.272 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:44.283 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:44.328 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:44.327+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:44.386 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:44.386 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:44.394 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:44.435 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:44.434+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:44.478 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:44.525 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:44.524+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:44.539 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:44.539 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:44.548 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:44.600 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:44.601 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:44.619 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:44.619 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:44.626 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:44.625+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:44.635 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:44.635 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:44.653 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:44.653 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:44.670 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:44.670 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:44.687 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:44.687 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790554696000001, 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.697 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:54:44.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.701 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790534159999993, 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.702 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:54:44.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:54:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:54:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:54:44.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790504161000001, 2024-09-14T22:54:44.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:44.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:54:44.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790473008000006, 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.713 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:44.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790450351999993, 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.718 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:54:44.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790429850999999, 2024-09-14T22:54:44.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:54:44.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:54:44.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790410089999995, 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.727 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790389305000005, 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:54:44.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790356892999995, 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.736 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:54:44.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790335494000004, 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.741 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:44.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.745 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790274095000001, 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:54:44.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790234182000006, 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.750 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:44.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.754 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790204426000003, 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:54:44.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:44.759 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790174191000006, 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:54:44.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:54:44.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790149451999994, 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.764 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:54:44.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790126021999995, 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.769 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:44.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:54:44.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790030286000004, 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.774 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:54:44.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.790001364000005, 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:54:44.778 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.782 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.789978039000005, 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.783 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:54:44.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:44.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:44.787 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.657280296000003, 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:44.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:54:44.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:54:44.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:44.792 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:44.794 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:44.794 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:44.794 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:44.795 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:44.796 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:44.796 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:44.793+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:44.894 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:44.894+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:44.911 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:44.911 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:44.911 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:44.911 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:44.920 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:44.995 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:44.995+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:45.023 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:45.023 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:45.023 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:45.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:45.097 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:45.095+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:45.134 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116114 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5500928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.078698574000001, 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053096953000000002, 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116114 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054015+0000", 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054183+0000", 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000168782 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054187+0000", 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1609999999999999e-06 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054194+0000", 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8550000000000002e-06 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054201+0000", 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0859999999999998e-06 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.106960+0000", 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759319999999998 2024-09-14T22:54:45.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107053+0000", 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3239999999999995e-05 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107112+0000", 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8508999999999998e-05 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116115 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5619712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.078503482000002, 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053139731000000003, 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116115 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.140 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054210+0000", 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054330+0000", 2024-09-14T22:54:45.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011983 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054332+0000", 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7010000000000001e-06 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054340+0000", 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2169999999999997e-06 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054353+0000", 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3804e-05 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107155+0000", 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052802031999999999 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107241+0000", 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5995999999999995e-05 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107350+0000", 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108151 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116116 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5509120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.078350692000001, 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053217990999999999, 2024-09-14T22:54:45.144 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116116 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054363+0000", 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054541+0000", 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017863699999999999 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054545+0000", 2024-09-14T22:54:45.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.427e-06 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054550+0000", 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4489999999999998e-06 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054560+0000", 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5380000000000008e-06 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107391+0000", 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052831271999999999 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107482+0000", 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0834000000000005e-05 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107581+0000", 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8833999999999996e-05 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116117 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5627904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.078145669999998, 2024-09-14T22:54:45.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053242065999999998, 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116117 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054568+0000", 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054807+0000", 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023963899999999999 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054811+0000", 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6320000000000001e-06 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054820+0000", 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054827+0000", 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.627e-06 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107623+0000", 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052795543 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107709+0000", 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6119999999999995e-05 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107810+0000", 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100577 2024-09-14T22:54:45.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116118 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.077875004000006, 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053147548000000003, 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.154 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116118 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.054838+0000", 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055075+0000", 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023718799999999999 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055080+0000", 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055086+0000", 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3709999999999996e-06 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055091+0000", 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5109999999999997e-06 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107855+0000", 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052763931 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107931+0000", 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6268999999999998e-05 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.107986+0000", 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4639999999999999e-05 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.158 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116119 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5636096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.077612844000001, 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053124491000000003, 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116119 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055100+0000", 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055228+0000", 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012761000000000001 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055232+0000", 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0249999999999996e-06 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055242+0000", 2024-09-14T22:54:45.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6209999999999999e-06 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055249+0000", 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0600000000000002e-06 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108028+0000", 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052779435999999999 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108117+0000", 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9029000000000004e-05 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108225+0000", 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010771 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.163 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116120 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5644288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.077453992000002, 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053095136000000001, 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116120 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.164 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055259+0000", 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055402+0000", 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014287999999999999 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055416+0000", 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.361e-05 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055425+0000", 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.38e-06 2024-09-14T22:54:45.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055432+0000", 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3189999999999997e-06 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108258+0000", 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052825940000000002 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108302+0000", 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832999999999999e-05 2024-09-14T22:54:45.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108354+0000", 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2173999999999999e-05 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116121 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5525504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:45.168 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.077267195999994, 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053136715000000001, 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116121 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055446+0000", 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055616+0000", 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016994499999999999 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055620+0000", 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5260000000000002e-06 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055627+0000", 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.187e-06 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055634+0000", 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5660000000000001e-06 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108394+0000", 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052759735000000002 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108480+0000", 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5649999999999995e-05 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108583+0000", 2024-09-14T22:54:45.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103106 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116122 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5652480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.077064950999997, 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053145748999999999, 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.173 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116122 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055648+0000", 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055826+0000", 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017742800000000001 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055830+0000", 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.997e-06 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055838+0000", 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2709999999999992e-06 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055845+0000", 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8050000000000001e-06 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108606+0000", 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052761246999999997 2024-09-14T22:54:45.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108689+0000", 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3403000000000001e-05 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108794+0000", 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104598 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116123 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5533696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.076856593000002, 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053173997000000001, 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116123 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.178 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.055857+0000", 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056134+0000", 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027731099999999997 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056139+0000", 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7129999999999996e-06 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056150+0000", 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1228e-05 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056158+0000", 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3850000000000002e-06 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108838+0000", 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052680034000000001 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.108930+0000", 2024-09-14T22:54:45.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2134999999999998e-05 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109031+0000", 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100191 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116124 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5660672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:45.182 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.076542876000005, 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053071471000000002, 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116124 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:45.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056170+0000", 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056352+0000", 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018171100000000001 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056356+0000", 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7740000000000002e-06 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056364+0000", 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8299999999999996e-06 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056368+0000", 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4839999999999998e-06 2024-09-14T22:54:45.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109055+0000", 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052686406999999998 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109139+0000", 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4420999999999997e-05 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109242+0000", 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010284400000000001 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116125 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.076337113999998, 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053094809999999999, 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.187 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116125 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056376+0000", 2024-09-14T22:54:45.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056511+0000", 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013528600000000001 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056515+0000", 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4630000000000001e-06 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056522+0000", 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.092e-06 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056535+0000", 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2785999999999999e-05 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109285+0000", 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052749934999999998 2024-09-14T22:54:45.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109371+0000", 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6166000000000001e-05 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109471+0000", 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010008199999999999 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116126 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5668864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.076168237999994, 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053154194000000002, 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116126 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.192 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056545+0000", 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056792+0000", 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024726400000000002 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056796+0000", 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1810000000000003e-06 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056804+0000", 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3880000000000002e-06 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056808+0000", 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9360000000000001e-06 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109508+0000", 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052700450000000003 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109600+0000", 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1448999999999998e-05 2024-09-14T22:54:45.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109699+0000", 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9525999999999995e-05 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116127 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5550080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.075897544, 2024-09-14T22:54:45.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053127120999999999, 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116127 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:45.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056816+0000", 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056936+0000", 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120277 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056940+0000", 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8940000000000003e-06 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056949+0000", 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1549999999999996e-06 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056955+0000", 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9000000000000003e-06 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109744+0000", 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052789124999999999 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109832+0000", 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8399999999999994e-05 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109943+0000", 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011037 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.200 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116128 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5558272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.075748269000002, 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053112611999999997, 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.201 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116128 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.056965+0000", 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057173+0000", 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020816199999999999 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057178+0000", 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4109999999999998e-06 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057188+0000", 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0475000000000001e-05 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057195+0000", 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6499999999999999e-06 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.109966+0000", 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052771277999999998 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110012+0000", 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5633000000000002e-05 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110078+0000", 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6002999999999998e-05 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:116129 3.14 3:281791be:::rbd_data.1135aa5cee52.0000000000000009:c [read 5677056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.075507027, 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053097755000000003, 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.205 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 116129 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057206+0000", 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057392+0000", 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018599600000000001 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057397+0000", 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4649999999999996e-06 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057408+0000", 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1399e-05 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.057415+0000", 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1359999999999999e-06 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110120+0000", 2024-09-14T22:54:45.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052705224000000002 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110208+0000", 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7274999999999994e-05 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.110304+0000", 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6260000000000003e-05 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:125033 3.3 3:c3ada8ea:::rbd_data.1135d503d0a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1753825~6423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.766816994999999, 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.064419233000000006, 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 125033 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365896+0000", 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.365899+0000", 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4329999999999998e-06 2024-09-14T22:54:45.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382895+0000", 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016995844 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382927+0000", 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2670999999999999e-05 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.382937+0000", 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5019999999999998e-06 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383022+0000", 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4902999999999997e-05 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383139+0000", 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000117769 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.383426+0000", 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000286474 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.418058+0000", 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.034632089999999997 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430245+0000", 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012187408 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430284+0000", 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8509e-05 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.430315+0000", 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.163e-05 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153281 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'291, mlcod=72'289)", 2024-09-14T22:54:45.215 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 41.697307533, 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060467871999999999, 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435406+0000", 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435411+0000", 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5589999999999999e-06 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435419+0000", 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.909e-06 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435423+0000", 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5140000000000003e-06 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435458+0000", 2024-09-14T22:54:45.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965000000000002e-05 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435470+0000", 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1559999999999999e-05 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495827+0000", 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060356819999999999 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495874+0000", 2024-09-14T22:54:45.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7003999999999999e-05 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4405.0:153282 3.11 e72/24 3:88b841d7:::rbd_object_map.1135d0236bc5:head v 72'292, mlcod=72'289)", 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 41.696675816999999, 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.060246112999999997, 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:45.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436037+0000", 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436038+0000", 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436042+0000", 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8920000000000002e-06 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495165+0000", 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059122945000000003 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495179+0000", 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4185000000000001e-05 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495346+0000", 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016677400000000001 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.495373+0000", 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7172e-05 2024-09-14T22:54:45.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496245+0000", 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00087206100000000004 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496284+0000", 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8525000000000001e-05 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:187784 3.2 3:45bacf70:::rbd_data.11354c7c4fd6.0000000000000028:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.301515096999999, 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.91484284000000005, 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 187784 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.225 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831198+0000", 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831199+0000", 2024-09-14T22:54:45.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2200000000000002e-07 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831203+0000", 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1019999999999999e-06 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831213+0000", 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0087e-05 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831218+0000", 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2299999999999999e-06 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831275+0000", 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6200999999999999e-05 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:17.831286+0000", 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0838e-05 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.744712+0000", 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.91342615800000004 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.744890+0000", 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000178488 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 5", 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745032+0000", 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014174500000000001 2024-09-14T22:54:45.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745748+0000", 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00071614400000000005 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.745984+0000", 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00023544500000000001 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.746015+0000", 2024-09-14T22:54:45.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1683000000000001e-05 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:18.746041+0000", 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5797000000000001e-05 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.231 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:45.233 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:45.233 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:45.231+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:45.288 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:45.288 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:54:45.288 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T22:54:45.288 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:54:45.293 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:54:45.333 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:45.331+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:45.376 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:45.376 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:45.376 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:45.376 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:45.385 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:45.434 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:45.433+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:45.489 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:45.489 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:45.489 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:45.489 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:45.489 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:45.498 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:45.538 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:45.537+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 111, 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:54:45.272415+0000", 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:52:49.483913+0000", 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:52:40.095883+0000", 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:54:45.577 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5801984, 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "noscrub", 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:54:45.578 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:45.579 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:54:45.580 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:45.581 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:45.582 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.583 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:54:45.584 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:45.585 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:45.586 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:45.587 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:45.588 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:45.589 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:45.590 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 2.9900000095367432, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 2.9900000095367432, 2024-09-14T22:54:45.591 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:54:45.592 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:54:45.593 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "109", 2024-09-14T22:54:45.594 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 47, 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 109, 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:54:45.595 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:54:45.596 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.597 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 2, 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 2, 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 1, 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:45.598 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.5721282958984375, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 110, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:45.599 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.602 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.603 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:54:45.604 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.0684051513671875, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 58, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.605 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.606 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.607 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:54:45.609 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:45.610 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 109, 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:45.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:45.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:45.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:54:45.614 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:54:45.615 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:45.616 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 95, 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:54:45.617 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.618 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.619 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.620 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 86, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:45.622 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.625 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.626 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:54:45.627 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 95, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:45.630 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.631 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.632 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:45.633 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 44, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:54:45.634 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:54:45.635 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.636 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:54:45.637 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:54:45.638 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:54:45.639 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 75, 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.640 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.641 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.642 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:54:45.643 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:45.646 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:45.647 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:45.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:54:45.649 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:54:45.650 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "2.6", 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: 5, 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-09-14T22:54:45.651 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.2", 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: 4, 2024-09-14T22:54:45.652 INFO:teuthology.orchestra.run.smithi076.stdout: 5 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.7", 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: 7, 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.8", 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:45.653 INFO:teuthology.orchestra.run.smithi076.stdout: 3, 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [ 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.3", 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: "mappings": [ 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: "from": 4, 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: "to": 2 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.654 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.12", 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: "mappings": [ 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: "from": 0, 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: "to": 7 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:54:45.655 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [ 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.3", 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: 0 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:54:45.656 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [ 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "snaps": [ 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "begin": 46, 2024-09-14T22:54:45.657 INFO:teuthology.orchestra.run.smithi076.stdout: "length": 2 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [], 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:54:45.658 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:54:45.659 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:45.659 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:45.662 INFO:tasks.thrashosds.thrasher:j is {'epoch': 111, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:54:45.272415+0000', 'last_up_change': '2024-09-14T22:52:49.483913+0000', 'last_in_change': '2024-09-14T22:52:40.095883+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 109, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5721282958984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 110, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0684051513671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 109, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [5, 2]}, {'pgid': '3.0', 'osds': [6, 7]}, {'pgid': '3.2', 'osds': [4, 5]}, {'pgid': '3.7', 'osds': [7, 6]}, {'pgid': '3.8', 'osds': [3, 6]}], 'pg_upmap_items': [{'pgid': '3.3', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.12', 'mappings': [{'from': 0, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.3', 'osds': [0]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 46, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:54:45.662 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:54:45.662 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:45.658+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.744160034000004, 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:45.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.118365914999998, 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:54:45.670 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:54:45.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.674 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.118277922999994, 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.675 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.679 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.118044006000005, 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:54:45.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:54:45.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:54:45.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.685 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.117863111000005, 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.686 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:54:45.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.690 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.117563607999998, 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:54:45.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:54:45.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.117217397999994, 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.695 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:54:45.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.116853669999998, 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.700 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.116264052999995, 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:54:45.704 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:54:45.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:54:45.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.116061815999998, 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.709 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:45.713 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.115872005, 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:45.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:54:45.717 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.115560001999995, 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.718 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:54:45.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:54:45.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:45.722 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.115317515000001, 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:54:45.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:54:45.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.115158553000001, 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.727 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.114524282999994, 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.732 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:54:45.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.114310173000007, 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:54:45.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:54:45.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.114173703000006, 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.741 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.113921223000006, 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.745 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:54:45.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.015022899000002, 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:54:45.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:54:45.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:45.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:54:45.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.5195362699999997, 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:54:45.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:54:45.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:54:45.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.762 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:45.763 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:45.763 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:45.763 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:45.765 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:45.763+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:45.865 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:45.863+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:45.867 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:45.867 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:45.868 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:45.868 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:45.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:45.967 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:45.964+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:45.978 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:45.978 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:45.979 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:45.979 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:45.979 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:45.987 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:46.039 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:54:46.039 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:54:46.051 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-14T22:54:45.275086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1835445365,"num_objects":2913,"num_object_clones":533,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":1808,"num_read":248312,"num_read_kb":1056082,"num_write":17111,"num_write_kb":704759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":77603230,"num_keys_recovered":54,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15632,"ondisk_log_size":15632,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1074992,"kb_used_data":702748,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":753899728,"statfs":{"total":773094113280,"available":771993321472,"internally_reserved":0,"allocated":719613952,"data_stored":832578672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-122942457,"num_objects":-78,"num_object_clones":-35,"num_object_copies":-156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-78,"num_whiteouts":-21,"num_read":6078,"num_read_kb":4481,"num_write":431,"num_write_kb":126241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.474564"},"pg_stats":[{"pgid":"3.1f","version":"108'577","reported_seq":7799,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275220+0000","last_change":"2024-09-14T22:54:43.285907+0000","last_active":"2024-09-14T22:54:44.275220+0000","last_peered":"2024-09-14T22:54:44.275220+0000","last_clean":"2024-09-14T22:54:44.275220+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:54:44.275220+0000","last_undegraded":"2024-09-14T22:54:44.275220+0000","last_fullsized":"2024-09-14T22:54:44.275220+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:52:23.370674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060071459999999997,"stat_sum":{"num_bytes":46312109,"num_objects":92,"num_object_clones":22,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":55,"num_read":8895,"num_read_kb":30904,"num_write":659,"num_write_kb":7678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.1e","version":"109'446","reported_seq":8159,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.562642+0000","last_change":"2024-09-14T22:54:43.291154+0000","last_active":"2024-09-14T22:54:44.562642+0000","last_peered":"2024-09-14T22:54:44.562642+0000","last_clean":"2024-09-14T22:54:44.562642+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:54:44.562642+0000","last_undegraded":"2024-09-14T22:54:44.562642+0000","last_fullsized":"2024-09-14T22:54:44.562642+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:54.872077+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.012408189,"stat_sum":{"num_bytes":31467283,"num_objects":95,"num_object_clones":23,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":57,"num_read":8341,"num_read_kb":20182,"num_write":423,"num_write_kb":12532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.1d","version":"109'424","reported_seq":8701,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560388+0000","last_change":"2024-09-14T22:54:43.293785+0000","last_active":"2024-09-14T22:54:44.560388+0000","last_peered":"2024-09-14T22:54:44.560388+0000","last_clean":"2024-09-14T22:54:44.560388+0000","last_became_active":"2024-09-14T22:53:06.762097+0000","last_became_peered":"2024-09-14T22:53:06.762097+0000","last_unstale":"2024-09-14T22:54:44.560388+0000","last_undegraded":"2024-09-14T22:54:44.560388+0000","last_fullsized":"2024-09-14T22:54:44.560388+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:39:10.230100+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.018652815999999999,"stat_sum":{"num_bytes":79015506,"num_objects":102,"num_object_clones":21,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":59,"num_read":9013,"num_read_kb":38470,"num_write":390,"num_write_kb":52005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.1c","version":"108'509","reported_seq":2700,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560224+0000","last_change":"2024-09-14T22:54:43.275529+0000","last_active":"2024-09-14T22:54:44.560224+0000","last_peered":"2024-09-14T22:54:44.560224+0000","last_clean":"2024-09-14T22:54:44.560224+0000","last_became_active":"2024-09-14T22:53:06.762804+0000","last_became_peered":"2024-09-14T22:53:06.762804+0000","last_unstale":"2024-09-14T22:54:44.560224+0000","last_undegraded":"2024-09-14T22:54:44.560224+0000","last_fullsized":"2024-09-14T22:54:44.560224+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:01:42.105579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065789099999999997,"stat_sum":{"num_bytes":53783990,"num_objects":75,"num_object_clones":13,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":45,"num_read":3778,"num_read_kb":11530,"num_write":570,"num_write_kb":8962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.1b","version":"109'285","reported_seq":4903,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275431+0000","last_change":"2024-09-14T22:54:43.293853+0000","last_active":"2024-09-14T22:54:44.275431+0000","last_peered":"2024-09-14T22:54:44.275431+0000","last_clean":"2024-09-14T22:54:44.275431+0000","last_became_active":"2024-09-14T22:53:06.758449+0000","last_became_peered":"2024-09-14T22:53:06.758449+0000","last_unstale":"2024-09-14T22:54:44.275431+0000","last_undegraded":"2024-09-14T22:54:44.275431+0000","last_fullsized":"2024-09-14T22:54:44.275431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:29:02.410999+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.014524155,"stat_sum":{"num_bytes":94331703,"num_objects":81,"num_object_clones":12,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":49,"num_read":4582,"num_read_kb":53980,"num_write":280,"num_write_kb":18743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.1a","version":"109'520","reported_seq":13346,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275265+0000","last_change":"2024-09-14T22:54:43.288310+0000","last_active":"2024-09-14T22:54:44.275265+0000","last_peered":"2024-09-14T22:54:44.275265+0000","last_clean":"2024-09-14T22:54:44.275265+0000","last_became_active":"2024-09-14T22:53:06.756674+0000","last_became_peered":"2024-09-14T22:53:06.756674+0000","last_unstale":"2024-09-14T22:54:44.275265+0000","last_undegraded":"2024-09-14T22:54:44.275265+0000","last_fullsized":"2024-09-14T22:54:44.275265+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:01:37.029458+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015485917,"stat_sum":{"num_bytes":59227272,"num_objects":115,"num_object_clones":28,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":68,"num_read":15124,"num_read_kb":52521,"num_write":442,"num_write_kb":29070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.19","version":"109'389","reported_seq":15647,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.562604+0000","last_change":"2024-09-14T22:54:43.283307+0000","last_active":"2024-09-14T22:54:44.562604+0000","last_peered":"2024-09-14T22:54:44.562604+0000","last_clean":"2024-09-14T22:54:44.562604+0000","last_became_active":"2024-09-14T22:53:51.942663+0000","last_became_peered":"2024-09-14T22:53:51.942663+0000","last_unstale":"2024-09-14T22:54:44.562604+0000","last_undegraded":"2024-09-14T22:54:44.562604+0000","last_fullsized":"2024-09-14T22:54:44.562604+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:01:21.493124+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0062308299999999997,"stat_sum":{"num_bytes":91642260,"num_objects":104,"num_object_clones":17,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":67,"num_read":15709,"num_read_kb":92366,"num_write":357,"num_write_kb":38459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.18","version":"109'627","reported_seq":2020,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.562662+0000","last_change":"2024-09-14T22:54:43.283321+0000","last_active":"2024-09-14T22:54:44.562662+0000","last_peered":"2024-09-14T22:54:44.562662+0000","last_clean":"2024-09-14T22:54:44.562662+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:54:44.562662+0000","last_undegraded":"2024-09-14T22:54:44.562662+0000","last_fullsized":"2024-09-14T22:54:44.562662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:46.168053+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0062282650000000002,"stat_sum":{"num_bytes":37099346,"num_objects":86,"num_object_clones":10,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":59,"num_read":2918,"num_read_kb":22712,"num_write":831,"num_write_kb":10476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.17","version":"109'568","reported_seq":6410,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560326+0000","last_change":"2024-09-14T22:54:43.282340+0000","last_active":"2024-09-14T22:54:44.560326+0000","last_peered":"2024-09-14T22:54:44.560326+0000","last_clean":"2024-09-14T22:54:44.560326+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:54:44.560326+0000","last_undegraded":"2024-09-14T22:54:44.560326+0000","last_fullsized":"2024-09-14T22:54:44.560326+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:50.768322+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0073334890000000003,"stat_sum":{"num_bytes":37689800,"num_objects":80,"num_object_clones":11,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":46,"num_read":7654,"num_read_kb":34618,"num_write":762,"num_write_kb":27721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.16","version":"109'349","reported_seq":9172,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560330+0000","last_change":"2024-09-14T22:54:43.278724+0000","last_active":"2024-09-14T22:54:44.560330+0000","last_peered":"2024-09-14T22:54:44.560330+0000","last_clean":"2024-09-14T22:54:44.560330+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:54:44.560330+0000","last_undegraded":"2024-09-14T22:54:44.560330+0000","last_fullsized":"2024-09-14T22:54:44.560330+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:15:11.062420+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0037448149999999999,"stat_sum":{"num_bytes":65188515,"num_objects":89,"num_object_clones":18,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":55,"num_read":9868,"num_read_kb":45593,"num_write":303,"num_write_kb":32667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.15","version":"108'352","reported_seq":8841,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563207+0000","last_change":"2024-09-14T22:54:43.280597+0000","last_active":"2024-09-14T22:54:44.563207+0000","last_peered":"2024-09-14T22:54:44.563207+0000","last_clean":"2024-09-14T22:54:44.563207+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:54:44.563207+0000","last_undegraded":"2024-09-14T22:54:44.563207+0000","last_fullsized":"2024-09-14T22:54:44.563207+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:57:40.214905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020158839999999999,"stat_sum":{"num_bytes":36426227,"num_objects":76,"num_object_clones":13,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":50,"num_read":9599,"num_read_kb":43048,"num_write":365,"num_write_kb":13556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.14","version":"108'497","reported_seq":10567,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275328+0000","last_change":"2024-09-14T22:54:43.290796+0000","last_active":"2024-09-14T22:54:44.275328+0000","last_peered":"2024-09-14T22:54:44.275328+0000","last_clean":"2024-09-14T22:54:44.275328+0000","last_became_active":"2024-09-14T22:53:06.758373+0000","last_became_peered":"2024-09-14T22:53:06.758373+0000","last_unstale":"2024-09-14T22:54:44.275328+0000","last_undegraded":"2024-09-14T22:54:44.275328+0000","last_fullsized":"2024-09-14T22:54:44.275328+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:38:25.991995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017911738999999999,"stat_sum":{"num_bytes":107682625,"num_objects":100,"num_object_clones":18,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":57,"num_read":11053,"num_read_kb":66136,"num_write":556,"num_write_kb":37505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563234+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:54:44.563234+0000","last_peered":"2024-09-14T22:54:44.563234+0000","last_clean":"2024-09-14T22:54:44.563234+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:54:44.563234+0000","last_undegraded":"2024-09-14T22:54:44.563234+0000","last_fullsized":"2024-09-14T22:54:44.563234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"108'580","reported_seq":7356,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275443+0000","last_change":"2024-09-14T22:54:43.280667+0000","last_active":"2024-09-14T22:54:44.275443+0000","last_peered":"2024-09-14T22:54:44.275443+0000","last_clean":"2024-09-14T22:54:44.275443+0000","last_became_active":"2024-09-14T22:53:06.757102+0000","last_became_peered":"2024-09-14T22:53:06.757102+0000","last_unstale":"2024-09-14T22:54:44.275443+0000","last_undegraded":"2024-09-14T22:54:44.275443+0000","last_fullsized":"2024-09-14T22:54:44.275443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:00:39.979523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024419770000000001,"stat_sum":{"num_bytes":103535924,"num_objects":83,"num_object_clones":15,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":48,"num_read":8926,"num_read_kb":41777,"num_write":686,"num_write_kb":24650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560289+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:54:44.560289+0000","last_peered":"2024-09-14T22:54:44.560289+0000","last_clean":"2024-09-14T22:54:44.560289+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:54:44.560289+0000","last_undegraded":"2024-09-14T22:54:44.560289+0000","last_fullsized":"2024-09-14T22:54:44.560289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"109'286","reported_seq":1701,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563398+0000","last_change":"2024-09-14T22:54:43.280961+0000","last_active":"2024-09-14T22:54:44.563398+0000","last_peered":"2024-09-14T22:54:44.563398+0000","last_clean":"2024-09-14T22:54:44.563398+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:54:44.563398+0000","last_undegraded":"2024-09-14T22:54:44.563398+0000","last_fullsized":"2024-09-14T22:54:44.563398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:10:32.468605+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0030059119999999999,"stat_sum":{"num_bytes":59782830,"num_objects":82,"num_object_clones":18,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":51,"num_read":1218,"num_read_kb":15237,"num_write":266,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563328+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:54:44.563328+0000","last_peered":"2024-09-14T22:54:44.563328+0000","last_clean":"2024-09-14T22:54:44.563328+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:54:44.563328+0000","last_undegraded":"2024-09-14T22:54:44.563328+0000","last_fullsized":"2024-09-14T22:54:44.563328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"108'736","reported_seq":13196,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.277122+0000","last_change":"2024-09-14T22:54:43.277638+0000","last_active":"2024-09-14T22:54:44.277122+0000","last_peered":"2024-09-14T22:54:44.277122+0000","last_clean":"2024-09-14T22:54:44.277122+0000","last_became_active":"2024-09-14T22:53:51.942267+0000","last_became_peered":"2024-09-14T22:53:51.942267+0000","last_unstale":"2024-09-14T22:54:44.277122+0000","last_undegraded":"2024-09-14T22:54:44.277122+0000","last_fullsized":"2024-09-14T22:54:44.277122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:55:19.215484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037017199999999999,"stat_sum":{"num_bytes":34254125,"num_objects":98,"num_object_clones":16,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":58,"num_read":15238,"num_read_kb":39819,"num_write":1008,"num_write_kb":19717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275379+0000","last_change":"2024-09-14T22:52:59.538529+0000","last_active":"2024-09-14T22:54:44.275379+0000","last_peered":"2024-09-14T22:54:44.275379+0000","last_clean":"2024-09-14T22:54:44.275379+0000","last_became_active":"2024-09-14T22:52:57.528089+0000","last_became_peered":"2024-09-14T22:52:57.528089+0000","last_unstale":"2024-09-14T22:54:44.275379+0000","last_undegraded":"2024-09-14T22:54:44.275379+0000","last_fullsized":"2024-09-14T22:54:44.275379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:11:33.379806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044267500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"109'357","reported_seq":15234,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560335+0000","last_change":"2024-09-14T22:54:43.290933+0000","last_active":"2024-09-14T22:54:44.560335+0000","last_peered":"2024-09-14T22:54:44.560335+0000","last_clean":"2024-09-14T22:54:44.560335+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:54:44.560335+0000","last_undegraded":"2024-09-14T22:54:44.560335+0000","last_fullsized":"2024-09-14T22:54:44.560335+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:13:00.167130+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015776776999999999,"stat_sum":{"num_bytes":79629348,"num_objects":95,"num_object_clones":16,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":55,"num_read":14218,"num_read_kb":48624,"num_write":317,"num_write_kb":19119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.3","version":"108'692","reported_seq":5614,"reported_epoch":110,"state":"remapped+peering","last_fresh":"2024-09-14T22:54:44.275655+0000","last_change":"2024-09-14T22:54:44.275655+0000","last_active":"2024-09-14T22:54:44.275655+0000","last_peered":"2024-09-14T22:54:42.328434+0000","last_clean":"2024-09-14T22:54:42.328434+0000","last_became_active":"2024-09-14T22:53:06.758167+0000","last_became_peered":"2024-09-14T22:53:06.758167+0000","last_unstale":"2024-09-14T22:54:44.275655+0000","last_undegraded":"2024-09-14T22:54:44.275655+0000","last_fullsized":"2024-09-14T22:54:44.275655+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058527299999999998,"stat_sum":{"num_bytes":81863904,"num_objects":107,"num_object_clones":21,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":68,"num_read":7951,"num_read_kb":56948,"num_write":787,"num_write_kb":40997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[{"start":"2f","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":197,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560232+0000","last_change":"2024-09-14T22:52:59.536698+0000","last_active":"2024-09-14T22:54:44.560232+0000","last_peered":"2024-09-14T22:54:44.560232+0000","last_clean":"2024-09-14T22:54:44.560232+0000","last_became_active":"2024-09-14T22:52:57.529227+0000","last_became_peered":"2024-09-14T22:52:57.529227+0000","last_unstale":"2024-09-14T22:54:44.560232+0000","last_undegraded":"2024-09-14T22:54:44.560232+0000","last_fullsized":"2024-09-14T22:54:44.560232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:13:04.163972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"109'318","reported_seq":1474,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563317+0000","last_change":"2024-09-14T22:54:43.291002+0000","last_active":"2024-09-14T22:54:44.563317+0000","last_peered":"2024-09-14T22:54:44.563317+0000","last_clean":"2024-09-14T22:54:44.563317+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:54:44.563317+0000","last_undegraded":"2024-09-14T22:54:44.563317+0000","last_fullsized":"2024-09-14T22:54:44.563317+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:38:43.907857+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0098377380000000004,"stat_sum":{"num_bytes":32696902,"num_objects":71,"num_object_clones":12,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":42,"num_read":1343,"num_read_kb":9458,"num_write":306,"num_write_kb":12045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":205,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.277086+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:54:44.277086+0000","last_peered":"2024-09-14T22:54:44.277086+0000","last_clean":"2024-09-14T22:54:44.277086+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:54:44.277086+0000","last_undegraded":"2024-09-14T22:54:44.277086+0000","last_fullsized":"2024-09-14T22:54:44.277086+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"109'540","reported_seq":8787,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275192+0000","last_change":"2024-09-14T22:54:43.285637+0000","last_active":"2024-09-14T22:54:44.275192+0000","last_peered":"2024-09-14T22:54:44.275192+0000","last_clean":"2024-09-14T22:54:44.275192+0000","last_became_active":"2024-09-14T22:53:06.760103+0000","last_became_peered":"2024-09-14T22:53:06.760103+0000","last_unstale":"2024-09-14T22:54:44.275192+0000","last_undegraded":"2024-09-14T22:54:44.275192+0000","last_fullsized":"2024-09-14T22:54:44.275192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:42:01.146170+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079375009999999996,"stat_sum":{"num_bytes":27748985,"num_objects":86,"num_object_clones":14,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":51,"num_read":9891,"num_read_kb":21165,"num_write":673,"num_write_kb":19185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563069+0000","last_change":"2024-09-14T22:53:00.415700+0000","last_active":"2024-09-14T22:54:44.563069+0000","last_peered":"2024-09-14T22:54:44.563069+0000","last_clean":"2024-09-14T22:54:44.563069+0000","last_became_active":"2024-09-14T22:52:57.528077+0000","last_became_peered":"2024-09-14T22:52:57.528077+0000","last_unstale":"2024-09-14T22:54:44.563069+0000","last_undegraded":"2024-09-14T22:54:44.563069+0000","last_fullsized":"2024-09-14T22:54:44.563069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:25.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"109'429","reported_seq":6619,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275170+0000","last_change":"2024-09-14T22:54:43.290766+0000","last_active":"2024-09-14T22:54:44.275170+0000","last_peered":"2024-09-14T22:54:44.275170+0000","last_clean":"2024-09-14T22:54:44.275170+0000","last_became_active":"2024-09-14T22:54:18.744069+0000","last_became_peered":"2024-09-14T22:54:18.744069+0000","last_unstale":"2024-09-14T22:54:44.275170+0000","last_undegraded":"2024-09-14T22:54:44.275170+0000","last_fullsized":"2024-09-14T22:54:44.275170+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:07:44.637710+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.018042861,"stat_sum":{"num_bytes":64670022,"num_objects":111,"num_object_clones":20,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":73,"num_read":6217,"num_read_kb":32194,"num_write":408,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"2.3","version":"20'1","reported_seq":196,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560234+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:54:44.560234+0000","last_peered":"2024-09-14T22:54:44.560234+0000","last_clean":"2024-09-14T22:54:44.560234+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:54:44.560234+0000","last_undegraded":"2024-09-14T22:54:44.560234+0000","last_fullsized":"2024-09-14T22:54:44.560234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":346,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563135+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:54:44.563135+0000","last_peered":"2024-09-14T22:54:44.563135+0000","last_clean":"2024-09-14T22:54:44.563135+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:54:44.563135+0000","last_undegraded":"2024-09-14T22:54:44.563135+0000","last_fullsized":"2024-09-14T22:54:44.563135+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'733","reported_seq":9174,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.562666+0000","last_change":"2024-09-14T22:54:43.284223+0000","last_active":"2024-09-14T22:54:44.562666+0000","last_peered":"2024-09-14T22:54:44.562666+0000","last_clean":"2024-09-14T22:54:44.562666+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:54:44.562666+0000","last_undegraded":"2024-09-14T22:54:44.562666+0000","last_fullsized":"2024-09-14T22:54:44.562666+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:34:35.716767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069150879999999998,"stat_sum":{"num_bytes":49431276,"num_objects":106,"num_object_clones":21,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":64,"num_read":13140,"num_read_kb":34957,"num_write":812,"num_write_kb":13071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.9","version":"109'451","reported_seq":1934,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.275203+0000","last_change":"2024-09-14T22:54:43.277233+0000","last_active":"2024-09-14T22:54:44.275203+0000","last_peered":"2024-09-14T22:54:44.275203+0000","last_clean":"2024-09-14T22:54:44.275203+0000","last_became_active":"2024-09-14T22:53:06.752666+0000","last_became_peered":"2024-09-14T22:53:06.752666+0000","last_unstale":"2024-09-14T22:54:44.275203+0000","last_undegraded":"2024-09-14T22:54:44.275203+0000","last_fullsized":"2024-09-14T22:54:44.275203+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:08:26.528655+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0044622710000000003,"stat_sum":{"num_bytes":66470125,"num_objects":100,"num_object_clones":17,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":69,"num_read":2353,"num_read_kb":9125,"num_write":410,"num_write_kb":21350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.a","version":"109'401","reported_seq":1793,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563187+0000","last_change":"2024-09-14T22:54:43.291140+0000","last_active":"2024-09-14T22:54:44.563187+0000","last_peered":"2024-09-14T22:54:44.563187+0000","last_clean":"2024-09-14T22:54:44.563187+0000","last_became_active":"2024-09-14T22:53:06.755021+0000","last_became_peered":"2024-09-14T22:53:06.755021+0000","last_unstale":"2024-09-14T22:54:44.563187+0000","last_undegraded":"2024-09-14T22:54:44.563187+0000","last_fullsized":"2024-09-14T22:54:44.563187+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:57:14.502646+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0103655,"stat_sum":{"num_bytes":33318939,"num_objects":71,"num_object_clones":10,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":45,"num_read":2704,"num_read_kb":8094,"num_write":421,"num_write_kb":4455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.b","version":"109'570","reported_seq":3414,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.562704+0000","last_change":"2024-09-14T22:54:43.288011+0000","last_active":"2024-09-14T22:54:44.562704+0000","last_peered":"2024-09-14T22:54:44.562704+0000","last_clean":"2024-09-14T22:54:44.562704+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:54:44.562704+0000","last_undegraded":"2024-09-14T22:54:44.562704+0000","last_fullsized":"2024-09-14T22:54:44.562704+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:12:10.567182+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010697808,"stat_sum":{"num_bytes":32098342,"num_objects":93,"num_object_clones":17,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":64,"num_read":5352,"num_read_kb":20048,"num_write":550,"num_write_kb":24595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.c","version":"108'466","reported_seq":5083,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.560253+0000","last_change":"2024-09-14T22:54:43.275631+0000","last_active":"2024-09-14T22:54:44.560253+0000","last_peered":"2024-09-14T22:54:44.560253+0000","last_clean":"2024-09-14T22:54:44.560253+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:54:44.560253+0000","last_undegraded":"2024-09-14T22:54:44.560253+0000","last_fullsized":"2024-09-14T22:54:44.560253+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:30:15.496315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000746577,"stat_sum":{"num_bytes":46091073,"num_objects":94,"num_object_clones":15,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":60,"num_read":5739,"num_read_kb":21623,"num_write":482,"num_write_kb":16955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.d","version":"109'545","reported_seq":6838,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563300+0000","last_change":"2024-09-14T22:54:43.280707+0000","last_active":"2024-09-14T22:54:44.563300+0000","last_peered":"2024-09-14T22:54:44.563300+0000","last_clean":"2024-09-14T22:54:44.563300+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:54:44.563300+0000","last_undegraded":"2024-09-14T22:54:44.563300+0000","last_fullsized":"2024-09-14T22:54:44.563300+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:04:09.405636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022468089999999998,"stat_sum":{"num_bytes":27687161,"num_objects":84,"num_object_clones":14,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":55,"num_read":8066,"num_read_kb":36235,"num_write":715,"num_write_kb":6566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.e","version":"108'415","reported_seq":6477,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563358+0000","last_change":"2024-09-14T22:54:43.279419+0000","last_active":"2024-09-14T22:54:44.563358+0000","last_peered":"2024-09-14T22:54:44.563358+0000","last_clean":"2024-09-14T22:54:44.563358+0000","last_became_active":"2024-09-14T22:53:06.761390+0000","last_became_peered":"2024-09-14T22:53:06.761390+0000","last_unstale":"2024-09-14T22:54:44.563358+0000","last_undegraded":"2024-09-14T22:54:44.563358+0000","last_fullsized":"2024-09-14T22:54:44.563358+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:57:42.946098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389686,"stat_sum":{"num_bytes":70922380,"num_objects":84,"num_object_clones":17,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":54,"num_read":7255,"num_read_kb":39924,"num_write":391,"num_write_kb":21317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.f","version":"107'284","reported_seq":6027,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563455+0000","last_change":"2024-09-14T22:54:43.280114+0000","last_active":"2024-09-14T22:54:44.563455+0000","last_peered":"2024-09-14T22:54:44.563455+0000","last_clean":"2024-09-14T22:54:44.563455+0000","last_became_active":"2024-09-14T22:53:06.759743+0000","last_became_peered":"2024-09-14T22:53:06.759743+0000","last_unstale":"2024-09-14T22:54:44.563455+0000","last_undegraded":"2024-09-14T22:54:44.563455+0000","last_fullsized":"2024-09-14T22:54:44.563455+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:47:20.588096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001981895,"stat_sum":{"num_bytes":37299790,"num_objects":88,"num_object_clones":19,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":54,"num_read":5710,"num_read_kb":21548,"num_write":252,"num_write_kb":12200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.10","version":"109'417","reported_seq":9331,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563386+0000","last_change":"2024-09-14T22:54:43.285553+0000","last_active":"2024-09-14T22:54:44.563386+0000","last_peered":"2024-09-14T22:54:44.563386+0000","last_clean":"2024-09-14T22:54:44.563386+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:54:44.563386+0000","last_undegraded":"2024-09-14T22:54:44.563386+0000","last_fullsized":"2024-09-14T22:54:44.563386+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:30:41.647633+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.004693348,"stat_sum":{"num_bytes":72449526,"num_objects":96,"num_object_clones":17,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":55,"num_read":10712,"num_read_kb":29840,"num_write":396,"num_write_kb":42111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.11","version":"106'518","reported_seq":5123,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563079+0000","last_change":"2024-09-14T22:54:43.278457+0000","last_active":"2024-09-14T22:54:44.563079+0000","last_peered":"2024-09-14T22:54:44.563079+0000","last_clean":"2024-09-14T22:54:44.563079+0000","last_became_active":"2024-09-14T22:53:06.759080+0000","last_became_peered":"2024-09-14T22:53:06.759080+0000","last_unstale":"2024-09-14T22:54:44.563079+0000","last_undegraded":"2024-09-14T22:54:44.563079+0000","last_fullsized":"2024-09-14T22:54:44.563079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:17:39.915215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061427900000000002,"stat_sum":{"num_bytes":68822428,"num_objects":83,"num_object_clones":16,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":48,"num_read":5940,"num_read_kb":34515,"num_write":651,"num_write_kb":24187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.12","version":"106'364","reported_seq":1225,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563168+0000","last_change":"2024-09-14T22:54:43.278928+0000","last_active":"2024-09-14T22:54:44.563168+0000","last_peered":"2024-09-14T22:54:44.563168+0000","last_clean":"2024-09-14T22:54:44.563168+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:54:44.563168+0000","last_undegraded":"2024-09-14T22:54:44.563168+0000","last_fullsized":"2024-09-14T22:54:44.563168+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:43:00.471209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094671299999999996,"stat_sum":{"num_bytes":47647596,"num_objects":82,"num_object_clones":13,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":60,"num_read":1229,"num_read_kb":10573,"num_write":406,"num_write_kb":9647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2e","length":"2"}]},{"pgid":"3.13","version":"108'867","reported_seq":6123,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T22:54:44.563119+0000","last_change":"2024-09-14T22:54:43.280078+0000","last_active":"2024-09-14T22:54:44.563119+0000","last_peered":"2024-09-14T22:54:44.563119+0000","last_clean":"2024-09-14T22:54:44.563119+0000","last_became_active":"2024-09-14T22:53:06.758354+0000","last_became_peered":"2024-09-14T22:53:06.758354+0000","last_unstale":"2024-09-14T22:54:44.563119+0000","last_undegraded":"2024-09-14T22:54:44.563119+0000","last_fullsized":"2024-09-14T22:54:44.563119+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:45:40.341510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017901639999999999,"stat_sum":{"num_bytes":58567666,"num_objects":100,"num_object_clones":19,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":67,"num_read":8460,"num_read_kb":12218,"num_write":1075,"num_write_kb":21635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2e","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1834854978,"num_objects":2909,"num_object_clones":533,"num_object_copies":5818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2909,"num_whiteouts":1808,"num_read":248196,"num_read_kb":1055982,"num_write":16950,"num_write_kb":702341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":978,"num_bytes_recovered":77603230,"num_keys_recovered":54,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":708026368,"data_stored":826513696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15512,"ondisk_log_size":15512,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145936,"kb_used_data":95608,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94225904,"statfs":{"total":96636764160,"available":96487325696,"internally_reserved":0,"allocated":97902592,"data_stored":113480242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85624,"kb_used_data":46108,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94286216,"statfs":{"total":96636764160,"available":96549085184,"internally_reserved":0,"allocated":47214592,"data_stored":55197839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114972,"kb_used_data":86876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94256868,"statfs":{"total":96636764160,"available":96519032832,"internally_reserved":0,"allocated":88961024,"data_stored":98214013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":219160,"kb_used_data":164560,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94152680,"statfs":{"total":96636764160,"available":96412344320,"internally_reserved":0,"allocated":168509440,"data_stored":190829532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135272,"kb_used_data":91564,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94236568,"statfs":{"total":96636764160,"available":96498245632,"internally_reserved":0,"allocated":93761536,"data_stored":102890558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56992,"kb_used_data":21640,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94314848,"statfs":{"total":96636764160,"available":96578404352,"internally_reserved":0,"allocated":22159360,"data_stored":31863115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":120908,"kb_used_data":81392,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94250932,"statfs":{"total":96636764160,"available":96512954368,"internally_reserved":0,"allocated":83345408,"data_stored":106371815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":196128,"kb_used_data":115000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94175712,"statfs":{"total":96636764160,"available":96435929088,"internally_reserved":0,"allocated":117760000,"data_stored":133731558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":115871744,"data_stored":132532950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":82034688,"data_stored":105757462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20865024,"data_stored":31254875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":92467200,"data_stored":102282318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":167202816,"data_stored":190215198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":87662592,"data_stored":97605754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45907968,"data_stored":54583505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":96014336,"data_stored":112281634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:54:46.053 INFO:tasks.thrashosds.thrasher:Setting 3.13 to [6, 2, 0, 3] 2024-09-14T22:54:46.053 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.13', '6', '2', '0', '3'] 2024-09-14T22:54:46.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.13 6 2 0 3 2024-09-14T22:54:46.068 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:46.066+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:46.092 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:46.092 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:46.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:46.092 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:46.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.092 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.092 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:46.092 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.008339426999996, 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:54:46.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.008121531, 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.097 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:54:46.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.005437885000006, 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.102 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:54:46.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.005279470000005, 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.106 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:54:46.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:54:46.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.005252204000001, 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.111 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:46.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.005045062999997, 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.115 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:54:46.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.005010799000004, 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:54:46.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:54:46.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.003378753000007, 2024-09-14T22:54:46.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:46.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:54:46.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.003354141000003, 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.129 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.133 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.003219854999998, 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:54:46.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.002067961999998, 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.142 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.002044057999996, 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:54:46.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:54:46.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.002010537999993, 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.147 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.151 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.001855891999995, 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.152 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:54:46.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.001814718000006, 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:54:46.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.001786558000006, 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.161 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:46.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 84.001597167, 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:54:46.165 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:54:46.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.085507340000007, 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:54:46.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:46.174 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.085487509000004, 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:46.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:54:46.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.8857811839999998, 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:54:46.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:54:46.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:54:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:54:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.185 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.185 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:46.187 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:46.187 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:46.185+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:46.287 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:46.286+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:46.288 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:54:46.338 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:46.338 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:46.347 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:46.388 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:46.387+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:46.488 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:46.486+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:46.489 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:46.489 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:46.489 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:46.489 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:46.490 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:46.500 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:46.589 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:46.588+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 114.09610132100001, 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:54:46.606 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:54:46.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:46.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.231443466000002, 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.613 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:54:46.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.231337530000005, 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.618 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:54:46.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.231262829000002, 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.622 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:54:46.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.626 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.231177348000003, 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:54:46.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:54:46.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.231030962000006, 2024-09-14T22:54:46.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:54:46.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.230980962000004, 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.636 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:54:46.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:54:46.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.230817798000004, 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.640 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.644 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.230779487999996, 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:54:46.648 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.230645819000003, 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:46.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:54:46.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.230565252000005, 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:54:46.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:54:46.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:54:46.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:46.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.230468314999996, 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:54:46.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.230416759999997, 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.663 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:54:46.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:54:46.667 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.227292130999999, 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:54:46.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.227160518000005, 2024-09-14T22:54:46.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:54:46.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.226874105999997, 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.677 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:46.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.681 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.168837852000003, 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:54:46.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.687 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.168798023999997, 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:46.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:54:46.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.168782008999997, 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:54:46.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:54:46.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:54:46.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.699 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.119653653999997, 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:46.700 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:54:46.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:46.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:54:46.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:54:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:46.706 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:46.706 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:46.707 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:46.706+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:46.708 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:46.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:46.807 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:46.806+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:46.815 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:46.815 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:46.815 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:46.815 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:46.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:46.908 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:46.906+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:46.920 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:46.921 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:46.938 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:46.939 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:46.955 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:46.971 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:46.972 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:46.972 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:46.972 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:46.981 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:46.988 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:46.989 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:47.006 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:47.006 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:47.009 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:47.008+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:47.023 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:47.023 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:47.040 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:47.040 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:47.094 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:47.094 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:47.102 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:47.110 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:47.109+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:47.205 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:47.205 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:47.211 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:47.209+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:47.213 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:47.295 INFO:teuthology.orchestra.run.smithi076.stderr:set 3.13 pg_upmap_items mapping to [6->2,0->3] 2024-09-14T22:54:47.312 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:47.309+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:47.366 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:47.367 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:47.375 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:47.419 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:47.417+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:47.477 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:47.478 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:47.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:47.514 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:47.513+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:47.615 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:47.612+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:47.629 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:47.629 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:47.638 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:47.716 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:47.715+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:47.788 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:47.789 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:47.798 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:47.841 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:47.839+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:47.900 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:47.901 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:47.909 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:47.918 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:47.917+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:48.019 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:48.017+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:48.053 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:48.053 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:48.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:48.120 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:48.118+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:48.164 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:48.165 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:48.173 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:48.221 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:48.220+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:48.307 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:54:48.318 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:48.318 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:48.321 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:48.320+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:48.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:48.422 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:48.419+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:48.443 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:48.443 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:48.453 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:48.523 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:48.522+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:48.557 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:48.557 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:48.566 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:48.650 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:48.649+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:48.710 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:48.710 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:48.718 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:48.748 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:54:48.760 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:48.758+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:48.826 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:48.824+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:48.870 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:48.870 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:48.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:48.963 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:48.962+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:49.022 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:49.022 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:49.027 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:49.026+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:49.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:49.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:49.128 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:49.126+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:49.173 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:49.174 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:49.174 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:49.174 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:49.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:49.191 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:49.191 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:49.207 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:49.207 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:49.222 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:49.223 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:49.228 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:49.226+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:49.239 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:49.239 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:49.255 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:49.255 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:49.272 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:49.288 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:49.288 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:49.327 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:49.327 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:49.327 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:49.327 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:49.336 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:49.340 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:49.338+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:49.430 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:49.429+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:49.438 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:49.438 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:49.438 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:49.438 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:49.438 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:49.447 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:49.531 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:49.530+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:49.592 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:49.592 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.508617137000002, 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.593 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:54:49.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.597 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.508399241000006, 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:54:49.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.602 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.505715594999998, 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.603 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:54:49.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:54:49.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:49.607 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.505557179999997, 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:54:49.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.505529913999993, 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.505322773000003, 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.617 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:54:49.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:54:49.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.505288508999996, 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.621 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:54:49.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.503656462999999, 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.626 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:54:49.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.630 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.503631850999994, 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:54:49.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:54:49.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.503497565000004, 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:54:49.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.502345672000004, 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:54:49.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.502321768000002, 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.644 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:54:49.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:54:49.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.502288247999999, 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.649 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:54:49.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:54:49.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.502133602000001, 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:54:49.653 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:54:49.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:54:49.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.502092427999997, 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.658 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.662 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.502064267999998, 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:54:49.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:54:49.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.667 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.501874877000006, 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:49.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.585785049999998, 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:54:49.672 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:54:49.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.585765218999995, 2024-09-14T22:54:49.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:54:49.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3860588939999996, 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:49.682 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:54:49.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:54:49.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:49.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:49.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:49.688 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:49.690 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:49.690 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:49.688+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:49.791 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:49.789+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:49.849 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:49.849 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:49.849 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:49.849 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:49.858 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:49.892 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:49.890+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:49.993 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:49.991+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:50.001 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:50.001 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:50.002 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:50.002 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:50.002 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:50.011 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:50.094 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:50.092+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:50.115 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:50.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:54:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 117.605366785, 2024-09-14T22:54:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:54:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.116 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:50.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:54:50.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:50.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:54:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:54:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.740708929999997, 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:54:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:54:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:54:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:54:50.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.740602994, 2024-09-14T22:54:50.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.134 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.740528292999997, 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.135 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:54:50.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.139 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.740442811999998, 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:54:50.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:54:50.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:54:50.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:50.144 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.740296426, 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:50.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:54:50.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:54:50.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:54:50.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:50.149 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.740246425999999, 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:54:50.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:54:50.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:54:50.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.740083261999999, 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:54:50.155 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:50.160 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.740044952000005, 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:54:50.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.739911282999998, 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:54:50.165 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:54:50.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:54:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.172 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.739830716, 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:54:50.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:54:50.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:50.177 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.739733779000005, 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:50.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:54:50.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:54:50.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.739682224000006, 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.183 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:54:50.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:54:50.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.188 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.736557594999994, 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:54:50.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:50.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:54:50.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:54:50.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.194 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.736425982, 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.195 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:54:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:50.199 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.736139570000006, 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:54:50.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:50.204 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.678103315999998, 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:50.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:50.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.678063487999999, 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:54:50.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:54:50.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.678047472999999, 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.217 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:54:50.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:54:50.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:54:50.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.224 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 45.628919117999999, 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:54:50.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:50.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:50.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:54:50.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:54:50.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:50.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:54:50.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:54:50.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:50.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:54:50.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:54:50.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:54:50.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:54:50.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:50.232 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:50.234 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:50.232+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:50.234 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:50.334 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:50.332+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:50.338 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:50.338 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:50.338 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:50.338 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:50.347 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:50.435 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:50.433+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:50.492 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:50.492 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:50.492 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:50.492 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:50.492 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:50.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:50.536 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:50.534+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:50.637 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:50.634+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:50.644 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:50.644 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:50.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:50.644 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:50.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.644 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.644 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:50.644 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.792469800999996, 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:54:50.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.791532211000003, 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:54:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:54:50.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.652 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:54:50.653 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.791440847999993, 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.654 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:54:50.656 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.657 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.658 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.790795962999994, 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.659 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:54:50.660 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.661 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.662 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.790308584000002, 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.663 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.664 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.665 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:54:50.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:54:50.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.666 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.666 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.666 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:54:50.666 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:54:50.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.667 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.668 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.789421309000005, 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.669 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.670 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:54:50.671 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.672 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:54:50.673 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.674 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.788828885000001, 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:54:50.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.676 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.677 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.678 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.679 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.680 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.788096069000005, 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.681 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.682 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:54:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.684 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.685 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.787996919999998, 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.686 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:50.687 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.688 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:54:50.689 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.787678462000002, 2024-09-14T22:54:50.690 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.691 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.692 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:54:50.693 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.694 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.787168968000003, 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.695 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:54:50.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:54:50.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:54:50.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.700 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.787111108999994, 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:54:50.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.786318485999999, 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.706 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:54:50.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.785643046999994, 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:54:50.712 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:54:50.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.785509736999998, 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.717 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:50.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.784892814000003, 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.722 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:50.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.784825956999995, 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.726 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:50.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:54:50.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:54:50.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.731 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.731 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.731 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.784375146000002, 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:50.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:54:50.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:54:50.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.736 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.784254912999998, 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:54:50.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.741 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.782936785999993, 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:50.742 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:50.743 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:54:50.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.743 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:50.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:50.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:50.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:50.747 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:50.748 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:50.749 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:50.750 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:50.747+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:50.764 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:54:50.850 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:50.847+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:50.904 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:50.904 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:50.904 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:50.904 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:50.914 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:50.950 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:50.949+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:51.052 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:51.049+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:51.059 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:51.059 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:51.059 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:51.059 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:51.059 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:51.068 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:51.153 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:51.151+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:51.213 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.213 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.214175771000001, 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:54:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:54:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:51.218 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.214156732999996, 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:54:51.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.223 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.214120777000005, 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:54:51.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.214099657000006, 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.228 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:54:51.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:54:51.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.232 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.212892742999998, 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:54:51.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:51.237 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.211063296000006, 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:51.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:54:51.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.210509009000006, 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.242 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:54:51.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.208524291000003, 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.247 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:54:51.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.251 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.208171970999999, 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:54:51.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.207888527999998, 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.256 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.207600431000003, 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:54:51.261 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:54:51.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:54:51.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.207225602999998, 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.266 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:51.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:54:51.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.206904788000003, 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:54:51.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:54:51.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.193219553000006, 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.276 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:54:51.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:51.280 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.192884743999997, 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:54:51.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.192554582, 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:51.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:54:51.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.192333796, 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.290 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:54:51.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:54:51.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.192064494999997, 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:54:51.294 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:51.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:54:51.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.191877399000006, 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:54:51.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:54:51.303 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.191559240000004, 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:51.304 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:51.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:54:51.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:51.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:51.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:54:51.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:54:51.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.308 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.308 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:51.308 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:51.308 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:51.309 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:51.310 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:51.311 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:51.308+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:51.410 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:51.407+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:51.457 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:51.457 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:51.457 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:51.457 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:51.466 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:51.473 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:51.473 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:51.490 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:51.490 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:51.507 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:51.507 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:51.510 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:51.509+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:51.523 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:51.523 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:51.540 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:51.557 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:51.558 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:51.575 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:51.575 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:51.577 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:51.577 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:51.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:51.593 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:51.648 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:51.646+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:51.695 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:51.696 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:51.704 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:51.712 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:51.710+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:51.766 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:54:51.813 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:51.811+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:51.860 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:51.861 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:51.870 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:51.914 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:51.913+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:51.974 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:51.974 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:51.983 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:52.025 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:52.023+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:52.116 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:52.114+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:52.127 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:52.127 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:52.136 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:52.216 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:52.215+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:52.253 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:52.253 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:52.262 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:52.317 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:52.317 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:54:52.317 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T22:54:52.317 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 1.000000 2024-09-14T22:54:52.318 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T22:54:52.318 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:52.316+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:52.407 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:52.407 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:52.415 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:52.418 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:52.417+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:52.517 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:52.518 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:52.519 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:52.517+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:52.526 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:52.620 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:52.618+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:52.669 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:52.669 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:52.678 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:52.722 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:52.721+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:52.744 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T22:54:52.784 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:52.784 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:52.793 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:52.834 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:52.833+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:52.922 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:52.921+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:52.939 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:52.939 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:52.948 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:53.023 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:53.022+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:53.059 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:53.142 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:53.141+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:53.203 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:53.203 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:53.211 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:53.225 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:53.224+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:53.326 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:53.325+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:53.354 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:53.354 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:53.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:53.427 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:53.426+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:53.466 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:53.466 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:53.474 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:53.528 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:53.527+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:53.577 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:53.577 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:53.586 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:53.606 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:53.650 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:53.649+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:53.728 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:53.728 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:53.729 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:53.728+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:53.732 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:53.733 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:53.737 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:53.750 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:53.751 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:53.767 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:53.768 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:53.784 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:54:53.792 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:53.793 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:53.824 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:53.824 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:53.830 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:53.829+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:53.843 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:53.843 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:53.861 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:53.861 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:210180 3.10 e120/24)", 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:53.867781+0000", 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 4.5794999999999998e-05, 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5504000000000002e-05, 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:53.869 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "queued for pg", 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867781+0000", 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867781+0000", 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867782+0000", 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8299999999999997e-07 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867787+0000", 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2700000000000004e-06 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867798+0000", 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1151999999999999e-05 2024-09-14T22:54:53.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867804+0000", 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.331e-06 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:210182 3.1f 3:f937d991:::rbd_data.11358204802c.000000000000001b:head [delete] snapc 27=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:53.867718+0000", 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 0.000108639, 2024-09-14T22:54:53.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019039799999999999, 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 210182 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:53.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867718+0000", 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867718+0000", 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867719+0000", 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.269e-06 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867725+0000", 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6160000000000001e-06 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867738+0000", 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2924e-05 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867745+0000", 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2989999999999999e-06 2024-09-14T22:54:53.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:53.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:53.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:53.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:53.867879+0000", 2024-09-14T22:54:53.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000133437 2024-09-14T22:54:53.876 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:53.876 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:53.876 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:53.876 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:53.877 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:54:53.877 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 2 2024-09-14T22:54:53.877 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:53.883 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:53.888 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:53.889 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:53.942 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:53.941+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:53.995 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:53.995 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:53.995 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:53.995 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:53.996 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:54.004 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:54.032 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:54.031+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:54.133 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:54.132+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:54.150 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.298584122999998, 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.151 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:54.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.155 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.297646533000005, 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:54:54.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.297555169999995, 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.160 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:54.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:54:54.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:54:54.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:54.164 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.296910284999996, 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:54:54.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.296422906000004, 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:54:54.169 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:54:54.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.173 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.295535631000007, 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:54:54.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:54:54.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.294943207000003, 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:54:54.178 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.182 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.294210391000007, 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.183 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.187 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.294111242, 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:54:54.191 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.293792784000004, 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.192 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:54:54.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:54:54.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.293283290000005, 2024-09-14T22:54:54.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:54:54.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.200 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.293225430999996, 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.201 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:54:54.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:54.205 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.292432808000001, 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:54:54.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:54:54.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.291757368999995, 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.210 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.214 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.291624059, 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:54:54.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:54:54.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.291007136000005, 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.219 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:54:54.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.290940278999997, 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.224 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.290489468000004, 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:54:54.228 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:54:54.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.232 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.290369235, 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:54:54.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:54:54.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:54:54.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.289051107999995, 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:54:54.237 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:54:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.241 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.242 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.242 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:54.243 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:54.244 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:54.242+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:54.344 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:54.343+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:54.388 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:54.388 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:54.388 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:54.388 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:54.397 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:54.480 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:54.479+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:54.540 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:54.540 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:54.540 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:54.540 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:54.540 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:54.546 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:54.544+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:54.549 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:54.647 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:54.645+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:54.694 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:54.694 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.695534705, 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.695 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.696 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:54:54.697 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.698 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.695515666999995, 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:54:54.700 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.701 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:54:54.702 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.703 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:54:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:54:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:54:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:54:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.704 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.695479711000004, 2024-09-14T22:54:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:54:54.705 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:54.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:54:54.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:54.710 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.695458591000005, 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:54:54.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:54:54.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.715 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.694251676999997, 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:54:54.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.692422230000005, 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.720 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:54:54.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.724 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.691867943000005, 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.725 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.689883225000003, 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.730 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:54:54.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:54.734 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.689530904999998, 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:54.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:54:54.738 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.689247461999997, 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.739 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.688959365000002, 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.744 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:54:54.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.688584536999997, 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.748 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:54:54.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.688263722000002, 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.753 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:54:54.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.674578487000005, 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.757 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:54:54.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.674243677999996, 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.762 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:54:54.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.673913515999999, 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.767 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:54:54.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:54:54.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.673692729999999, 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:54:54.771 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:54:54.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.775 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.673423428999996, 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.776 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:54:54.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:54:54.781 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.781 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.781 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.781 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.673236333000006, 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:54:54.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:54:54.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:54:54.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:54:54.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:54:54.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.787 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.787 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:54.787 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:54.787 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 77.672918174000003, 2024-09-14T22:54:54.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:54.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:54:54.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.791 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:54.792 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:54.792 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:54.794 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:54.794 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:54:54.794 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:54.792+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:54.894 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:54.892+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:54.941 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:54.941 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:54.941 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:54.941 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:54.950 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:54.995 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:54.994+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:55.053 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:55.054 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:55.054 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:55.054 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:55.054 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:55.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:55.097 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:55.095+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:55.167 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:55.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.956555342000001, 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:54:55.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.832893568000003, 2024-09-14T22:54:55.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:55.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:54:55.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:54:55.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 65.901126097000002, 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:54:55.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:54:55.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.181 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.991179854999999, 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:54:55.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172324 3.4 3:213525dd:::rbd_data.113598f99d1a.00000000000000b5:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 40.978389845000002, 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051499340000000001, 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.186 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172324 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186688+0000", 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1829999999999999e-06 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186693+0000", 2024-09-14T22:54:55.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4100000000000001e-06 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186705+0000", 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2169e-05 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186712+0000", 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9789999999999996e-06 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186758+0000", 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6687000000000002e-05 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186839+0000", 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-05 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186980+0000", 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000141569 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191467+0000", 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0044866840000000003 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191734+0000", 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026690299999999998 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191793+0000", 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9447000000000002e-05 2024-09-14T22:54:55.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191837+0000", 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3433000000000002e-05 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.042519212000002, 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:54:55.192 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:54:55.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:54:55.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.586442335000001, 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.198 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:55.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.202 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.586415013, 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.203 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:54:55.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.586003521999999, 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:54:55.207 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.584344255000001, 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:54:55.211 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:54:55.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:54:55.215 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.580312831000001, 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194089 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.579725164999999, 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005182201, 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194089 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585353+0000", 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8899999999999998e-07 2024-09-14T22:54:55.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585357+0000", 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8470000000000003e-06 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585369+0000", 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1992e-05 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585377+0000", 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6090000000000003e-06 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590237+0000", 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048596910000000002 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590352+0000", 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000115387 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590534+0000", 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018168599999999999 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194119 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.578489805, 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051579030000000001, 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.225 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194119 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8200000000000001e-07 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586590+0000", 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586596+0000", 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.212e-06 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586601+0000", 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8289999999999998e-06 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591330+0000", 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047292439999999996 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591418+0000", 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8448000000000004e-05 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591745+0000", 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00032651799999999998 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.229 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.578469354999999, 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:54:55.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:54:55.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.578449680999999, 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.234 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:55.238 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.578411027000001, 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:55.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.242 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 28.578391147000001, 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.243 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:54:55.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 24.205219954, 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:54:55.247 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:54:55.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:54:55.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:55.253 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 24.204218659999999, 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:54:55.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:55.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:54:55.259 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 22.898162468999999, 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:54:55.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:54:55.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:54:55.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.265 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:55.267 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:55.267 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:55.265+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:55.367 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:55.365+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:55.411 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:55.411 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:55.412 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:55.412 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:55.420 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:55.468 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:55.466+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:55.562 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:55.610 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:55.611 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:55.611 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:55.611 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:55.612 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:55.613 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:55.611+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:55.713 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:55.712+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:55.757 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:55.757 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:55.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:55.757 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:55.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.758 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.758 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:55.758 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850570527000002, 2024-09-14T22:54:55.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:54:55.758 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:54:55.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850549990999994, 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.763 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:54:55.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:54:55.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:54:55.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.768 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850519992000002, 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.769 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:54:55.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.773 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850488838999993, 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:54:55.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850466182999995, 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.779 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:54:55.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850445682, 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.784 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:54:55.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:54:55.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:54:55.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.789 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850425920999996, 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:54:55.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:54:55.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.794 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850405136000006, 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:54:55.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.799 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850372723999996, 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:54:55.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850351325000005, 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.804 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:54:55.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.808 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:54:55.809 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:54:55.809 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.809 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.809 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.809 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.809 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:55.812 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850289926000002, 2024-09-14T22:54:55.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:54:55.812 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.812 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:55.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:54:55.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850250012999993, 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:54:55.817 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:54:55.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850220257000004, 2024-09-14T22:54:55.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850190022000007, 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.827 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:54:55.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:54:55.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.832 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850165282999995, 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.833 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:54:55.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.837 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850141852999997, 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:54:55.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:55.842 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850046117000005, 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:54:55.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:54:55.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.850017195000007, 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.847 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:54:55.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:54:55.851 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.849993870000006, 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.852 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:54:55.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.717296126999997, 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:55.857 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:55.858 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:55.858 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:55.858 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:54:55.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.858 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:55.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:55.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:55.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:55.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:55.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:55.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:55.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:54:55.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:54:55.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:55.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:54:55.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:54:55.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:55.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:54:55.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:54:55.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:55.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:54:55.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:54:55.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:54:55.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:54:55.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:54:55.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:54:55.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:55.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:55.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:55.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:54:55.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:54:55.865 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.865 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.866 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.866 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:55.866 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:55.866 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:55.868 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:54:55.868 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:55.866+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:55.902 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:54:55.968 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:55.966+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:55.978 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:55.978 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:55.978 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:55.978 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:55.987 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T22:54:56.028 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:56.029 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:56.045 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:56.045 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:56.061 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:56.061 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:56.068 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:56.067+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:56.077 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:56.077 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:56.090 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:56.090 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:56.091 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:56.091 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:56.091 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:56.093 INFO:teuthology.orchestra.run.smithi076.stdout:osd.4: {} 2024-09-14T22:54:56.093 INFO:teuthology.orchestra.run.smithi076.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T22:54:56.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T22:54:56.112 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:56.113 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:56.127 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:56.127 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:56.143 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:56.143 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:56.184 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:56.183+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:56.243 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:56.243 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:56.252 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:56.293 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:56.292+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:56.371 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:56.370+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:56.394 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:56.395 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:56.403 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:56.472 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:56.471+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:56.505 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:56.505 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:56.514 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:56.573 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:56.572+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:56.658 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:56.658 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:56.666 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:56.709 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:56.708+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:56.769 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:56.769 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:56.775 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:56.774+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:56.777 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:54:56.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:54:56.876 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:56.875+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:56.932 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:56.933 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:56.941 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:54:56.983 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:56.981+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:57.078 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:57.077+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:57.084 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:57.084 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:57.093 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:54:57.179 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:57.178+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:57.196 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:57.196 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:57.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:54:57.280 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:57.279+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:57.307 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:57.307 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:57.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:54:57.381 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:54:57.380+0000 7f1220370640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33805) UID: 0 2024-09-14T22:54:57.419 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:57.420 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:57.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:54:57.482 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:57.481+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:57.535 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:57.535 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:57.544 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:54:57.587 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:57.586+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:57.646 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:57.646 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:57.654 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:54:57.695 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:57.694+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:57.761 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:57.761 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:54:57.761 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:54:57.761 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T22:54:57.762 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:54:57.785 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:57.783+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:57.791 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T22:54:57.791 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-14T22:54:57.791 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:54:57.791 DEBUG:teuthology.orchestra.run.smithi114:> type ceph-objectstore-tool 2024-09-14T22:54:57.799 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:57.799 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:57.804 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:54:57.814 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:54:57.846 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T22:54:57.846 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-14T22:54:57.908 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:57.907+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:57.969 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:57.970 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:57.979 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:54:57.986 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:57.985+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:58.087 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:58.086+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:58.123 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:58.123 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:58.132 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:54:58.156 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:54:58.216 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:58.215+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:58.279 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:54:58.280 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:54:58.289 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:54:58.289 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:58.287+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:58.295 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:54:58.295 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:54:58.312 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:54:58.312 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:54:58.329 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:54:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:54:58.345 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:54:58.345 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:54:58.345 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:54:58.361 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:54:58.362 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:54:58.378 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:54:58.378 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:54:58.389 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:58.388+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:58.395 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:54:58.395 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:54:58.398 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:54:58.399 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:54:58.400 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:58.400 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:58.400 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:58.400 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:58.409 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:54:58.494 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:58.493+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:58.552 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:58.552 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:58.552 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:58.552 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:58.552 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:58.561 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:54:58.601 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:58.600+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:58.692 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:58.690+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:58.704 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 91.493882290000002, 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.705 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:54:58.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:54:58.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 78.370220516000003, 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:54:58.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:54:58.714 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 69.438453045000003, 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.715 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:54:58.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:54:58.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 44.528506802999999, 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.719 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:54:58.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:172324 3.4 3:213525dd:::rbd_data.113598f99d1a.00000000000000b5:head [delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 44.515716793000003, 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051499340000000001, 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 172324 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186687+0000", 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186688+0000", 2024-09-14T22:54:58.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1829999999999999e-06 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186693+0000", 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4100000000000001e-06 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186705+0000", 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2169e-05 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186712+0000", 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9789999999999996e-06 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186758+0000", 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6687000000000002e-05 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186839+0000", 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-05 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.186980+0000", 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000141569 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191467+0000", 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0044866840000000003 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191734+0000", 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026690299999999998 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191793+0000", 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9447000000000002e-05 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.191837+0000", 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3433000000000002e-05 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.729 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 35.579846160000002, 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:54:58.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:54:58.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:54:58.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.123769283000001, 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.736 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:54:58.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:54:58.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:54:58.739 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.123741961, 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.740 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.123330469999999, 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.744 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:54:58.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:54:58.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:58.748 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.121671202999998, 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:54:58.752 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.117639779000001, 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.753 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:54:58.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.757 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194089 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.117052113, 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005182201, 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194089 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585353+0000", 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8899999999999998e-07 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585357+0000", 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8470000000000003e-06 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585369+0000", 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1992e-05 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585377+0000", 2024-09-14T22:54:58.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6090000000000003e-06 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590237+0000", 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048596910000000002 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590352+0000", 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000115387 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590534+0000", 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018168599999999999 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194119 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.115816752999997, 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051579030000000001, 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194119 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8200000000000001e-07 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586590+0000", 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586596+0000", 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.212e-06 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586601+0000", 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8289999999999998e-06 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591330+0000", 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047292439999999996 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591418+0000", 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8448000000000004e-05 2024-09-14T22:54:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591745+0000", 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00032651799999999998 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.115796303000003, 2024-09-14T22:54:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:54:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.115776629000003, 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:54:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.777 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.115737975000002, 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:54:58.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 32.115718094999998, 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.782 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:54:58.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.742546902000001, 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.787 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.788 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:54:58.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:54:58.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.741545607999999, 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.793 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.794 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:54:58.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:58.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 26.435489416999999, 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:58.799 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:54:58.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:58.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:54:58.803 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:58.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:54:58.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:54:58.805 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.805 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.805 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.805 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:58.805 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:58.805 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:58.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:54:58.807 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:58.805+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:58.906 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:58.905+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:58.955 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:58.955 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:58.955 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:54:58.955 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:58.964 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:54:59.007 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:54:59.006+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:54:59.107 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:59.107 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:54:59.107 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:54:59.107 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:54:59.107 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:59.108 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:59.106+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:59.116 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:54:59.208 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:59.207+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:59.220 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:54:59.220 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:54:59.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313987769999997, 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.221 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:54:59.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313967234000003, 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.225 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:54:59.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:54:59.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313937234999997, 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.230 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:54:59.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.234 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313906082000003, 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.235 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.238 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.239 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313883426000004, 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:54:59.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:54:59.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313862924999995, 2024-09-14T22:54:59.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:54:59.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313843164000005, 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.249 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313822379000001, 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:54:59.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.257 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313789967000005, 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.258 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:54:59.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:59.262 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313768568, 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:54:59.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:54:59.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313707168999997, 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:54:59.267 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:54:59.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:54:59.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.272 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313667256000002, 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:54:59.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313637499999999, 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.277 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:54:59.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.281 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313607265000002, 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:54:59.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:54:59.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.286 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313582526000005, 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:54:59.290 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313559096000006, 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.291 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:54:59.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:59.295 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.31346336, 2024-09-14T22:54:59.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:54:59.296 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.296 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.296 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.296 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.296 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313434438000002, 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.301 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:54:59.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.313411113000001, 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:54:59.305 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:54:59.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:54:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 74.180713370000007, 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:54:59.310 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:54:59.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.314 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.315 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.315 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:54:59.315 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:54:59.315 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:54:59.316 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:54:59.317 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:59.315+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:54:59.417 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:54:59.415+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:54:59.423 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:59.424 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:59.424 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:54:59.424 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:59.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:54:59.518 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:54:59.516+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:54:59.602 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:59.602 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:54:59.602 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:54:59.602 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:54:59.603 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:59.613 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:54:59.618 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:54:59.617+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:54:59.720 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:54:59.718+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:54:59.760 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:54:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:54:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:54:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:54:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.862209874000001, 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:54:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:54:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.236415754999996, 2024-09-14T22:54:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.236327763000006, 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:54:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.236093846000003, 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:54:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.235912951000003, 2024-09-14T22:54:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:54:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:54:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.235613447999995, 2024-09-14T22:54:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:54:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:54:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.235267238000006, 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:54:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:54:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.234903509999995, 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:54:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:54:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:54:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:54:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:54:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.234313893000007, 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:54:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.234111655999996, 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.233921844999998, 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:54:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:54:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:54:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.233609842000007, 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:54:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.233367354999999, 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:54:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:54:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.233208392999998, 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:54:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:54:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.232574123000006, 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:54:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:54:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:54:59.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:54:59.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:54:59.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:59.839 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.232360013000005, 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:54:59.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:54:59.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.232223543000003, 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:59.848 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.231971063000003, 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:59.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:54:59.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.852 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.133072738999999, 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:54:59.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:54:59.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:54:59.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:54:59.860 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.637586110000001, 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.861 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:54:59.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:54:59.867 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.868 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.868 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.868 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:54:59.868 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:54:59.868 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:54:59.868 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:54:59.870 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:54:59.867+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:54:59.871 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:54:59.871 INFO:teuthology.orchestra.run.smithi114.stdout:3.f 2024-09-14T22:54:59.871 INFO:teuthology.orchestra.run.smithi114.stdout:3.e 2024-09-14T22:54:59.871 INFO:teuthology.orchestra.run.smithi114.stdout:3.9 2024-09-14T22:54:59.871 INFO:teuthology.orchestra.run.smithi114.stdout:3.2 2024-09-14T22:54:59.871 INFO:teuthology.orchestra.run.smithi114.stdout:3.1d 2024-09-14T22:54:59.871 INFO:teuthology.orchestra.run.smithi114.stdout:3.1c 2024-09-14T22:54:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:3.1a 2024-09-14T22:54:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:3.1 2024-09-14T22:54:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:3.a 2024-09-14T22:54:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:3.1b 2024-09-14T22:54:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:3.14 2024-09-14T22:54:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:3.13 2024-09-14T22:54:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:3.11 2024-09-14T22:54:59.970 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:54:59.969+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:00.014 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:00.015 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:00.015 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:00.015 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:00.023 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:00.071 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:00.069+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:00.127 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:00.127 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:00.127 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:00.135 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:00.171 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:00.170+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:00.273 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:00.271+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:00.281 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:00.281 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:00.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:00.281 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:00.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.281 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.281 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:00.281 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.197233967000003, 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:55:00.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.197016070999993, 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.286 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:55:00.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.290 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.194332424999999, 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.291 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:55:00.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.295 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.194174009999998, 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.296 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:55:00.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:00.301 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.194146743999994, 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:55:00.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.306 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.193939603000004, 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:00.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:55:00.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.193905338999997, 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:55:00.312 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:55:00.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:55:00.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.192273293, 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:00.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:55:00.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.192248680999995, 2024-09-14T22:55:00.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:55:00.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.328 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.192114395000004, 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.329 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:55:00.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:55:00.333 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.190962502000005, 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.334 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:55:00.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:55:00.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.190938598000002, 2024-09-14T22:55:00.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:55:00.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.344 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.190905078, 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:55:00.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:55:00.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:55:00.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:55:00.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:55:00.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:55:00.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.350 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.190750432000002, 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.190709257999998, 2024-09-14T22:55:00.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:55:00.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:55:00.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.361 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.190681097999999, 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:00.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:55:00.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:55:00.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 98.190491707000007, 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.367 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:55:00.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.274401879999999, 2024-09-14T22:55:00.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:00.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:55:00.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:55:00.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.274382048999996, 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.378 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:00.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:55:00.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.074675723999999, 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:55:00.383 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:00.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:55:00.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:55:00.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:00.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:00.390 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:00.392 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:00.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 3.e --file /var/log/ceph/exp.3.e.4 2024-09-14T22:55:00.394 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:00.390+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:00.400 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:55:00.493 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:00.492+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:00.520 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:00.521 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:00.537 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:00.537 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:00.538 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:00.538 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:00.538 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:00.538 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:00.546 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:00.553 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:00.553 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:00.569 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:00.569 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:00.569 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:00.586 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:00.586 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:00.593 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:00.592+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:00.604 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:00.604 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:00.619 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:00.619 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:00.622 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:00.623 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:00.651 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:00.651 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:00.660 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:00.706 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:00.704+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:00.764 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:00.764 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:00.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:00.825 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:55:00.828 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:00.826+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:00.894 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:00.895 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:00.896 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:00.895+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:00.904 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:00.997 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:00.996+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:01.048 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:01.049 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:01.058 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:01.098 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:01.096+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:01.199 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:01.198+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:01.202 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:01.202 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:01.211 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:01.300 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:01.298+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:01.356 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:01.356 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:01.365 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:01.409 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:01.408+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:01.468 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:01.469 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:01.477 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:01.518 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:01.517+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:01.602 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:01.601+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:01.620 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:01.620 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:01.629 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:01.703 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:01.702+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:01.732 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:01.732 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:01.741 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:01.803 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:01.802+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:01.896 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:01.896 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:01.904 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:01.903+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:01.906 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:02.005 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:02.004+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:02.051 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:02.052 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:02.060 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:02.074 INFO:teuthology.orchestra.run.smithi114.stderr:Exporting 3.e info 3.e( v 123'439 (0'0,123'439] local-lis/les=24/25 n=72 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) 2024-09-14T22:55:02.084 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:700315e5:::rbd_data.113528220df0.0000000000000001:head# 2024-09-14T22:55:02.084 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:701e3dcd:::rbd_id.image_client.0-clone25:head# 2024-09-14T22:55:02.085 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7053b998:::rbd_data.113535cec067.00000000000000d6:head# 2024-09-14T22:55:02.085 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7083726d:::rbd_data.1135ae724a7.000000000000000f:1c# 2024-09-14T22:55:02.088 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7083726d:::rbd_data.1135ae724a7.000000000000000f:head# 2024-09-14T22:55:02.088 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:70b4cba6:::rbd_data.1135a9d9ff07.000000000000029c:head# 2024-09-14T22:55:02.088 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:70cef686:::rbd_data.1135757ab9e2.00000000000000c6:head# 2024-09-14T22:55:02.088 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:71064c3b:::rbd_data.11356c4af504.000000000000001c:head# 2024-09-14T22:55:02.088 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:71201e33:::rbd_data.11354571ccdf.000000000000033c:head# 2024-09-14T22:55:02.088 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7141b74b:::rbd_data.1135757ab9e2.0000000000000017:head# 2024-09-14T22:55:02.088 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:71ab9f81:::rbd_data.113598f99d1a.000000000000001a:23# 2024-09-14T22:55:02.090 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:71ab9f81:::rbd_data.113598f99d1a.000000000000001a:head# 2024-09-14T22:55:02.090 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:720cc99a:::rbd_data.113535cec067.0000000000000113:head# 2024-09-14T22:55:02.090 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:723d6318:::rbd_data.113564f98e71.000000000000002e:head# 2024-09-14T22:55:02.090 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:724544bb:::rbd_data.113528220df0.000000000000015d:head# 2024-09-14T22:55:02.090 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:724f8d0e:::rbd_data.11356c4af504.0000000000000028:head# 2024-09-14T22:55:02.090 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:72bc0738:::rbd_data.1135757ab9e2.0000000000000009:head# 2024-09-14T22:55:02.092 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:72dd1ac7:::rbd_header.113526a7a752:head# 2024-09-14T22:55:02.092 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:72f44ec0:::rbd_data.1135d0236bc5.0000000000000054:head# 2024-09-14T22:55:02.092 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:73565c0e:::rbd_data.1135d9b82239.0000000000000052:head# 2024-09-14T22:55:02.092 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:736ddfb9:::rbd_data.1135757ab9e2.00000000000000b5:head# 2024-09-14T22:55:02.092 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:738d1a4e:::rbd_data.1135d0236bc5.000000000000004f:head# 2024-09-14T22:55:02.093 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:73a92c16:::rbd_data.113544222fa6.0000000000000015:head# 2024-09-14T22:55:02.093 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:73f5e937:::rbd_data.1135757ab9e2.0000000000000028:head# 2024-09-14T22:55:02.093 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:74037f3e:::rbd_data.1135aa5cee52.0000000000000002:head# 2024-09-14T22:55:02.094 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:741ca396:::rbd_data.113535cec067.0000000000000120:head# 2024-09-14T22:55:02.094 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:743003c7:::rbd_data.1135a9d9ff07.0000000000000268:head# 2024-09-14T22:55:02.094 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:744c4d28:::rbd_data.1135d9b82239.000000000000001f:head# 2024-09-14T22:55:02.095 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:745d0c46:::rbd_data.113528220df0.00000000000000a9:head# 2024-09-14T22:55:02.095 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:74c98c64:::rbd_data.1135a9d9ff07.00000000000002d8:head# 2024-09-14T22:55:02.096 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:74fabb3c:::rbd_data.113528220df0.000000000000012c:head# 2024-09-14T22:55:02.096 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:750b332c:::rbd_data.1135ae724a7.0000000000000015:1c# 2024-09-14T22:55:02.099 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:750b332c:::rbd_data.1135ae724a7.0000000000000015:head# 2024-09-14T22:55:02.099 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:75172921:::rbd_data.1135757ab9e2.00000000000000c8:5# 2024-09-14T22:55:02.099 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:75172921:::rbd_data.1135757ab9e2.00000000000000c8:head# 2024-09-14T22:55:02.099 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:753baf56:::rbd_data.1135f0c182e.0000000000000001:1d# 2024-09-14T22:55:02.113 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:753baf56:::rbd_data.1135f0c182e.0000000000000001:head# 2024-09-14T22:55:02.113 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:754a9820:::rbd_data.113528220df0.000000000000014f:head# 2024-09-14T22:55:02.113 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:755ba22d:::rbd_data.113528220df0.000000000000010a:head# 2024-09-14T22:55:02.113 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:75711201:::rbd_data.113535cec067.0000000000000103:8# 2024-09-14T22:55:02.113 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:75711201:::rbd_data.113535cec067.0000000000000103:head# 2024-09-14T22:55:02.113 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7582e6ff:::rbd_data.113535cec067.00000000000000d3:head# 2024-09-14T22:55:02.113 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:75a56acb:::rbd_data.113528220df0.0000000000000007:d# 2024-09-14T22:55:02.114 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:75a56acb:::rbd_data.113528220df0.0000000000000007:head# 2024-09-14T22:55:02.114 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:75c2281f:::rbd_data.1135a5fb387.000000000000004b:head# 2024-09-14T22:55:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:75fb7031:::rbd_data.1135a9d9ff07.00000000000002a0:head# 2024-09-14T22:55:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7630590d:::rbd_data.1135a9d9ff07.000000000000028d:head# 2024-09-14T22:55:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7666168a:::rbd_data.113535cec067.0000000000000184:8# 2024-09-14T22:55:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7666168a:::rbd_data.113535cec067.0000000000000184:head# 2024-09-14T22:55:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7667303c:::rbd_data.1135c0c4962c.0000000000000036:a# 2024-09-14T22:55:02.119 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7667303c:::rbd_data.1135c0c4962c.0000000000000036:head# 2024-09-14T22:55:02.119 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:767618fe:::rbd_data.113528220df0.0000000000000033:head# 2024-09-14T22:55:02.119 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7680bc5f:::rbd_header.1135a9fbf9fc:head# 2024-09-14T22:55:02.119 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:76988a02:::rbd_data.1135e35238c.0000000000000003:1f# 2024-09-14T22:55:02.121 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:76988a02:::rbd_data.1135e35238c.0000000000000003:head# 2024-09-14T22:55:02.123 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:76a899d6:::rbd_data.113535cec067.0000000000000146:head# 2024-09-14T22:55:02.123 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:76bca24b:::rbd_data.113535cec067.0000000000000005:head# 2024-09-14T22:55:02.125 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:76c92187:::rbd_data.113528220df0.0000000000000000:head# 2024-09-14T22:55:02.125 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head# 2024-09-14T22:55:02.125 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77163fad:::rbd_data.113528220df0.000000000000007c:head# 2024-09-14T22:55:02.125 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:775c16dd:::rbd_data.11354571ccdf.0000000000000356:head# 2024-09-14T22:55:02.125 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77757f95:::rbd_data.1135757ab9e2.00000000000000f7:head# 2024-09-14T22:55:02.126 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7795f047:::rbd_data.113535cec067.00000000000000c1:8# 2024-09-14T22:55:02.128 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:7795f047:::rbd_data.113535cec067.00000000000000c1:head# 2024-09-14T22:55:02.128 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77ae61dd:::rbd_data.113528220df0.000000000000008d:head# 2024-09-14T22:55:02.128 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77c3a9bd:::rbd_data.1135a9d9ff07.0000000000000265:head# 2024-09-14T22:55:02.128 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77c69cfa:::rbd_data.113528220df0.000000000000014a:head# 2024-09-14T22:55:02.128 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77ccf83b:::rbd_data.1135757ab9e2.0000000000000091:head# 2024-09-14T22:55:02.128 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77d56b77:::rbd_data.113528220df0.0000000000000053:head# 2024-09-14T22:55:02.128 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77f8496f:::rbd_data.113535cec067.00000000000000e2:8# 2024-09-14T22:55:02.129 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77f8496f:::rbd_data.113535cec067.00000000000000e2:head# 2024-09-14T22:55:02.129 INFO:teuthology.orchestra.run.smithi114.stderr:Read #3:77fd5285:::rbd_data.113530dd5d95.0000000000000007:head# 2024-09-14T22:55:02.129 INFO:teuthology.orchestra.run.smithi114.stderr:Export successful 2024-09-14T22:55:02.144 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:02.143+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:02.203 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:02.203 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:02.207 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:02.206+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:02.212 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:02.308 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:02.307+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:02.355 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:02.356 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:02.365 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:02.409 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:02.407+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:02.509 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:02.510 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:02.508+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:02.517 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:02.611 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:02.609+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:02.622 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:02.623 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:02.624 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:55:02.631 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:02.656 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 3.e 2024-09-14T22:55:02.737 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:02.737 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:02.737 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:02.736+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:02.744 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:02.744 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:02.746 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:02.767 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:02.767 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:02.785 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:02.785 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:02.804 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:02.804 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:02.804 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:02.812 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:02.811+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:02.823 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:02.823 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:02.842 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:55:02.842 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:02.843 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:02.861 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:02.861 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:02.864 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:02.865 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:02.898 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:55:02.898 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.000010148000001, 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:55:02.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:55:02.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:55:02.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:02.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.905 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.374216028999996, 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:55:02.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.374128037000006, 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:55:02.910 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:55:02.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:55:02.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.373894120000003, 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:02.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.373713225000003, 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.920 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:55:02.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.373413721999995, 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:55:02.924 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:55:02.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.928 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.373067512000006, 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.929 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.933 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.372703783999995, 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:55:02.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:55:02.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.372114167000007, 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.938 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:02.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.942 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.371911929999996, 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:55:02.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:55:02.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.371722118999998, 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.947 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:02.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.951 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.371410116000007, 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:55:02.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:55:02.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.371167628999999, 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.956 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:55:02.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.960 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.371008666999998, 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:55:02.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.370374397000006, 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:55:02.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.969 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.370160287000004, 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:55:02.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.370023817000003, 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:02.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.978 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 85.369771337000003, 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:55:02.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:55:02.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.270873012999999, 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.983 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:55:02.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:55:02.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:02.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.989 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.775386384000001, 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:02.990 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:55:02.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.995 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:02.996 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:02.996 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:02.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:02.998 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:02.996+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:03.098 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:03.096+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:03.142 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:03.142 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:03.142 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:03.150 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:03.234 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:03.233+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:03.293 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:03.294 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:03.294 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:03.294 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:03.294 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:03.300 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:03.299+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:03.303 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:03.401 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:03.400+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:03.462 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:03.462 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.37877689600001, 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.463 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:55:03.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:55:03.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:55:03.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:55:03.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:55:03.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.378559, 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:55:03.469 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:55:03.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:55:03.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.375875354, 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:55:03.474 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:55:03.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.375716939, 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.479 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:55:03.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:03.483 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.375689673, 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:03.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.487 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.37548253200001, 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.488 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.375448268, 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:55:03.492 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:55:03.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:55:03.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.373816222, 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:55:03.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:55:03.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.37379161, 2024-09-14T22:55:03.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:55:03.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.505 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.37365732400001, 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.506 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:55:03.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:03.510 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.37250543099999, 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:03.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:55:03.514 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.37248152700001, 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:55:03.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:03.519 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.372448007, 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:55:03.523 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.372293361, 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.524 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:55:03.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:55:03.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.372252187, 2024-09-14T22:55:03.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:03.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.532 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.372224027, 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.533 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:55:03.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:55:03.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 101.372034636, 2024-09-14T22:55:03.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:55:03.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.455944809000002, 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:55:03.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:03.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.455924977999999, 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.546 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.256218653000001, 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:55:03.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:55:03.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:03.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.556 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:03.558 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:03.558 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:03.556+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:03.658 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:03.657+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:03.661 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:03.662 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:03.662 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:03.662 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:03.673 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:03.759 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:03.758+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:03.816 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:03.816 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:03.817 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:03.825 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:03.848 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:55:03.868 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:03.867+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:03.940 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:03.940 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:03.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:03.940 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:03.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.940 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 131.43017494700001, 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:55:03.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:55:03.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.947 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.565517091999993, 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:55:03.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:55:03.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.565411155999996, 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.952 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.953 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.954 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.956 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.565336454999994, 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.960 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.565250973999994, 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:55:03.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:03.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.565104587999997, 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.965 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:55:03.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:03.969 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.565054587999995, 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:03.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:55:03.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.564891423999995, 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.974 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.978 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.564853114000002, 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:55:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:55:03.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.564719444999994, 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.983 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.987 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.564638877999997, 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:55:03.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.564541941000002, 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:55:03.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:55:03.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:03.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.564490386000003, 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:03.997 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:03.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:55:04.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:04.001 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.561365757000004, 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:04.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:04.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:04.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.561234143999997, 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.006 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:55:04.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 87.560947732000002, 2024-09-14T22:55:04.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:04.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:55:04.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.502911478000001, 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:04.015 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:55:04.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:55:04.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.020 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.502871650000003, 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:55:04.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.502855635000003, 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:04.026 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:04.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:04.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:55:04.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:04.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 59.453727280000003, 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:04.032 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:04.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:55:04.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:55:04.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:55:04.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:04.038 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:04.038 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:04.039 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:04.040 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:04.038+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:04.140 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:04.139+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:04.144 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:04.145 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:04.145 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:04.145 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:04.153 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:04.241 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:04.239+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:04.255 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:04.255 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:04.255 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:04.255 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:04.255 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:04.263 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:04.341 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:04.340+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:04.358 INFO:teuthology.orchestra.run.smithi114.stdout: marking collection for removal 2024-09-14T22:55:04.358 INFO:teuthology.orchestra.run.smithi114.stdout:setting '_remove' omap key 2024-09-14T22:55:04.358 INFO:teuthology.orchestra.run.smithi114.stdout:finish_remove_pgs 3.e_head removing 3.e 2024-09-14T22:55:04.363 INFO:teuthology.orchestra.run.smithi114.stdout:Remove successful 2024-09-14T22:55:04.368 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:04.368 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:04.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:04.368 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:04.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.368 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.368 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:04.368 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.516227149, 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:55:04.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.515289559, 2024-09-14T22:55:04.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:04.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:55:04.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:55:04.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.515198196, 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.378 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:55:04.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.382 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.514553311, 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.383 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:55:04.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.387 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.51406593199999, 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:55:04.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:55:04.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.513178657, 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.392 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:55:04.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.51258623299999, 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:55:04.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:55:04.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:55:04.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:04.401 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.511853417, 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:55:04.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.511754268, 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.406 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.410 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.51143580999999, 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:55:04.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.510926316, 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.415 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:55:04.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.510868457, 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:55:04.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:55:04.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.51007583400001, 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:55:04.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.509400395, 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:04.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.509267085, 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:55:04.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.508650162, 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.438 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:55:04.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.508583305, 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.443 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:04.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:55:04.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.50813249399999, 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:55:04.447 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:55:04.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:55:04.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.508012261, 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.506694134, 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:55:04.460 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.461 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.461 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.461 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.461 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.461 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:04.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:04.463 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:04.461+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:04.563 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:04.562+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:04.606 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:04.606 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:04.606 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:04.615 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:04.664 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:04.663+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:04.758 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:04.759 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:04.759 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:04.765 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:04.764+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:04.767 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:04.865 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:55:04.866 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:04.864+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:04.869 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.e.4 2024-09-14T22:55:04.924 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.925819168000004, 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.925 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:55:04.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:55:04.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.929 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.925800129999999, 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.930 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:55:04.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:04.935 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.925764173999994, 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.939 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.925743053999994, 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.940 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.924536140000001, 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.944 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:55:04.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:55:04.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.922706692999995, 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:04.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.922152405999995, 2024-09-14T22:55:04.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:55:04.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.920167688000006, 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.958 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.919815368000002, 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:55:04.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:55:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:55:04.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:04.967 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.919531925000001, 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:55:04.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.919243828000006, 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.972 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:04.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:55:04.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:04.976 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.918869000000001, 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.977 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:55:04.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:55:04.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.918548185000006, 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.981 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:55:04.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.985 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.904862949999995, 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:55:04.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:55:04.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.904528141, 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:55:04.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:55:04.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:55:04.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.904197979000003, 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:55:04.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.903977193000003, 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:55:04.998 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:04.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:55:05.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.903707892, 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.903520795999995, 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:05.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:55:05.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.010 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 87.903202637000007, 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:05.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:55:05.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:55:05.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.014 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:05.015 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:05.015 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:05.016 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:05.017 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:05.017 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:05.017 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:05.017 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:05.018 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:05.014+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:05.029 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:05.029 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:05.045 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:05.045 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:05.045 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:05.062 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:05.062 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:05.079 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:05.079 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:05.096 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:05.096 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:05.101 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:05.101 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:05.136 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:05.136 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:05.136 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:05.135+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:05.145 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:05.218 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:05.216+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:05.288 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:05.288 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:05.297 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:05.340 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:05.339+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:05.400 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:05.400 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:05.408 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:05.449 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:05.448+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:05.520 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:05.519+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:05.552 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:05.553 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:05.561 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:05.645 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:05.644+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:05.704 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:05.704 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:05.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:05.753 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:05.752+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:05.823 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:05.821+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:05.858 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:05.858 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:05.865 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:55:05.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:05.924 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:05.923+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:05.979 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:05.979 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:05.988 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:06.025 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:06.024+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:06.091 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:06.091 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:06.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:06.144 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:06.143+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:06.202 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:06.202 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:06.211 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:06.229 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:06.228+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:06.333 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:06.332+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:06.349 INFO:teuthology.orchestra.run.smithi114.stdout:get_pg_num_history pg_num_history pg_num_history(e123 pg_nums {1={16=1},2={19=8},3={24=32}} deleted_pools ) 2024-09-14T22:55:06.349 INFO:teuthology.orchestra.run.smithi114.stdout:Importing pgid 3.e 2024-09-14T22:55:06.349 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:700315e5:::rbd_data.113528220df0.0000000000000001:head# 2024-09-14T22:55:06.349 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.350 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:701e3dcd:::rbd_id.image_client.0-clone25:head# 2024-09-14T22:55:06.350 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 0=[]:{} 2024-09-14T22:55:06.351 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7053b998:::rbd_data.113535cec067.00000000000000d6:head# 2024-09-14T22:55:06.351 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{} 2024-09-14T22:55:06.351 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7083726d:::rbd_data.1135ae724a7.000000000000000f:1c# 2024-09-14T22:55:06.357 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:06.357 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:06.359 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7083726d:::rbd_data.1135ae724a7.000000000000000f:head# 2024-09-14T22:55:06.359 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 1c=[]:{1c=[1c]} 2024-09-14T22:55:06.360 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:70b4cba6:::rbd_data.1135a9d9ff07.000000000000029c:head# 2024-09-14T22:55:06.360 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 2f=[]:{} 2024-09-14T22:55:06.360 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:70cef686:::rbd_data.1135757ab9e2.00000000000000c6:head# 2024-09-14T22:55:06.360 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 5=[]:{} 2024-09-14T22:55:06.360 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:71064c3b:::rbd_data.11356c4af504.000000000000001c:head# 2024-09-14T22:55:06.360 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 25=[]:{} 2024-09-14T22:55:06.360 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:71201e33:::rbd_data.11354571ccdf.000000000000033c:head# 2024-09-14T22:55:06.361 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 22=[]:{} 2024-09-14T22:55:06.361 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7141b74b:::rbd_data.1135757ab9e2.0000000000000017:head# 2024-09-14T22:55:06.361 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 5=[]:{} 2024-09-14T22:55:06.361 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:71ab9f81:::rbd_data.113598f99d1a.000000000000001a:23# 2024-09-14T22:55:06.361 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:71ab9f81:::rbd_data.113598f99d1a.000000000000001a:head# 2024-09-14T22:55:06.361 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 23=[]:{23=[23]} 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:720cc99a:::rbd_data.113535cec067.0000000000000113:head# 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{} 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:723d6318:::rbd_data.113564f98e71.000000000000002e:head# 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:snapset b=[]:{} 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:724544bb:::rbd_data.113528220df0.000000000000015d:head# 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:724f8d0e:::rbd_data.11356c4af504.0000000000000028:head# 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 25=[]:{} 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:72bc0738:::rbd_data.1135757ab9e2.0000000000000009:head# 2024-09-14T22:55:06.363 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 5=[]:{} 2024-09-14T22:55:06.364 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:72dd1ac7:::rbd_header.113526a7a752:head# 2024-09-14T22:55:06.364 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 0=[]:{} 2024-09-14T22:55:06.364 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:72f44ec0:::rbd_data.1135d0236bc5.0000000000000054:head# 2024-09-14T22:55:06.364 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 1e=[]:{} 2024-09-14T22:55:06.364 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:73565c0e:::rbd_data.1135d9b82239.0000000000000052:head# 2024-09-14T22:55:06.364 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 0=[]:{} 2024-09-14T22:55:06.365 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:736ddfb9:::rbd_data.1135757ab9e2.00000000000000b5:head# 2024-09-14T22:55:06.365 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 5=[]:{} 2024-09-14T22:55:06.365 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:738d1a4e:::rbd_data.1135d0236bc5.000000000000004f:head# 2024-09-14T22:55:06.365 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 1e=[]:{} 2024-09-14T22:55:06.365 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:73a92c16:::rbd_data.113544222fa6.0000000000000015:head# 2024-09-14T22:55:06.366 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 0=[]:{} 2024-09-14T22:55:06.366 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:73f5e937:::rbd_data.1135757ab9e2.0000000000000028:head# 2024-09-14T22:55:06.367 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 5=[]:{} 2024-09-14T22:55:06.367 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:74037f3e:::rbd_data.1135aa5cee52.0000000000000002:head# 2024-09-14T22:55:06.367 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:06.368 INFO:teuthology.orchestra.run.smithi114.stdout:snapset c=[]:{} 2024-09-14T22:55:06.376 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:741ca396:::rbd_data.113535cec067.0000000000000120:head# 2024-09-14T22:55:06.376 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{} 2024-09-14T22:55:06.377 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:743003c7:::rbd_data.1135a9d9ff07.0000000000000268:head# 2024-09-14T22:55:06.377 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 2f=[]:{} 2024-09-14T22:55:06.377 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:744c4d28:::rbd_data.1135d9b82239.000000000000001f:head# 2024-09-14T22:55:06.377 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 0=[]:{} 2024-09-14T22:55:06.378 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:745d0c46:::rbd_data.113528220df0.00000000000000a9:head# 2024-09-14T22:55:06.378 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.378 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:74c98c64:::rbd_data.1135a9d9ff07.00000000000002d8:head# 2024-09-14T22:55:06.378 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 2f=[]:{} 2024-09-14T22:55:06.378 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:74fabb3c:::rbd_data.113528220df0.000000000000012c:head# 2024-09-14T22:55:06.378 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.378 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:750b332c:::rbd_data.1135ae724a7.0000000000000015:1c# 2024-09-14T22:55:06.387 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:750b332c:::rbd_data.1135ae724a7.0000000000000015:head# 2024-09-14T22:55:06.387 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 1c=[]:{1c=[1c]} 2024-09-14T22:55:06.387 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:75172921:::rbd_data.1135757ab9e2.00000000000000c8:5# 2024-09-14T22:55:06.388 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:75172921:::rbd_data.1135757ab9e2.00000000000000c8:head# 2024-09-14T22:55:06.388 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 5=[]:{5=[5]} 2024-09-14T22:55:06.388 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:753baf56:::rbd_data.1135f0c182e.0000000000000001:1d# 2024-09-14T22:55:06.434 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:06.432+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:06.438 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:753baf56:::rbd_data.1135f0c182e.0000000000000001:head# 2024-09-14T22:55:06.438 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 1d=[]:{1d=[1d]} 2024-09-14T22:55:06.439 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:754a9820:::rbd_data.113528220df0.000000000000014f:head# 2024-09-14T22:55:06.439 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.439 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:755ba22d:::rbd_data.113528220df0.000000000000010a:head# 2024-09-14T22:55:06.439 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.439 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:75711201:::rbd_data.113535cec067.0000000000000103:8# 2024-09-14T22:55:06.439 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:75711201:::rbd_data.113535cec067.0000000000000103:head# 2024-09-14T22:55:06.440 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{8=[8]} 2024-09-14T22:55:06.440 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7582e6ff:::rbd_data.113535cec067.00000000000000d3:head# 2024-09-14T22:55:06.440 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{} 2024-09-14T22:55:06.440 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:75a56acb:::rbd_data.113528220df0.0000000000000007:d# 2024-09-14T22:55:06.441 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:75a56acb:::rbd_data.113528220df0.0000000000000007:head# 2024-09-14T22:55:06.441 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{d=[d]} 2024-09-14T22:55:06.441 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:75c2281f:::rbd_data.1135a5fb387.000000000000004b:head# 2024-09-14T22:55:06.441 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 0=[]:{} 2024-09-14T22:55:06.443 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:75fb7031:::rbd_data.1135a9d9ff07.00000000000002a0:head# 2024-09-14T22:55:06.443 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 2f=[]:{} 2024-09-14T22:55:06.444 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7630590d:::rbd_data.1135a9d9ff07.000000000000028d:head# 2024-09-14T22:55:06.444 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 2f=[]:{} 2024-09-14T22:55:06.444 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7666168a:::rbd_data.113535cec067.0000000000000184:8# 2024-09-14T22:55:06.444 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7666168a:::rbd_data.113535cec067.0000000000000184:head# 2024-09-14T22:55:06.444 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{8=[8]} 2024-09-14T22:55:06.445 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7667303c:::rbd_data.1135c0c4962c.0000000000000036:a# 2024-09-14T22:55:06.449 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7667303c:::rbd_data.1135c0c4962c.0000000000000036:head# 2024-09-14T22:55:06.449 INFO:teuthology.orchestra.run.smithi114.stdout:snapset a=[]:{a=[a]} 2024-09-14T22:55:06.449 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:767618fe:::rbd_data.113528220df0.0000000000000033:head# 2024-09-14T22:55:06.450 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.450 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7680bc5f:::rbd_header.1135a9fbf9fc:head# 2024-09-14T22:55:06.450 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 0=[]:{} 2024-09-14T22:55:06.450 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:76988a02:::rbd_data.1135e35238c.0000000000000003:1f# 2024-09-14T22:55:06.456 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:76988a02:::rbd_data.1135e35238c.0000000000000003:head# 2024-09-14T22:55:06.457 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 1f=[]:{1f=[1f]} 2024-09-14T22:55:06.461 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:76a899d6:::rbd_data.113535cec067.0000000000000146:head# 2024-09-14T22:55:06.461 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{} 2024-09-14T22:55:06.461 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:76bca24b:::rbd_data.113535cec067.0000000000000005:head# 2024-09-14T22:55:06.461 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 0=[]:{} 2024-09-14T22:55:06.462 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:76c92187:::rbd_data.113528220df0.0000000000000000:head# 2024-09-14T22:55:06.462 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.462 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head# 2024-09-14T22:55:06.462 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 1e=[]:{} 2024-09-14T22:55:06.463 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77163fad:::rbd_data.113528220df0.000000000000007c:head# 2024-09-14T22:55:06.463 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.463 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:775c16dd:::rbd_data.11354571ccdf.0000000000000356:head# 2024-09-14T22:55:06.463 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 22=[]:{} 2024-09-14T22:55:06.463 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77757f95:::rbd_data.1135757ab9e2.00000000000000f7:head# 2024-09-14T22:55:06.463 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 5=[]:{} 2024-09-14T22:55:06.464 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7795f047:::rbd_data.113535cec067.00000000000000c1:8# 2024-09-14T22:55:06.464 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:7795f047:::rbd_data.113535cec067.00000000000000c1:head# 2024-09-14T22:55:06.464 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{8=[8]} 2024-09-14T22:55:06.465 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77ae61dd:::rbd_data.113528220df0.000000000000008d:head# 2024-09-14T22:55:06.465 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.465 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77c3a9bd:::rbd_data.1135a9d9ff07.0000000000000265:head# 2024-09-14T22:55:06.465 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 2f=[]:{} 2024-09-14T22:55:06.465 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77c69cfa:::rbd_data.113528220df0.000000000000014a:head# 2024-09-14T22:55:06.465 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.466 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77ccf83b:::rbd_data.1135757ab9e2.0000000000000091:head# 2024-09-14T22:55:06.466 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 5=[]:{} 2024-09-14T22:55:06.466 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77d56b77:::rbd_data.113528220df0.0000000000000053:head# 2024-09-14T22:55:06.466 INFO:teuthology.orchestra.run.smithi114.stdout:snapset d=[]:{} 2024-09-14T22:55:06.466 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77f8496f:::rbd_data.113535cec067.00000000000000e2:8# 2024-09-14T22:55:06.466 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77f8496f:::rbd_data.113535cec067.00000000000000e2:head# 2024-09-14T22:55:06.467 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 8=[]:{8=[8]} 2024-09-14T22:55:06.467 INFO:teuthology.orchestra.run.smithi114.stdout:Write #3:77fd5285:::rbd_data.113530dd5d95.0000000000000007:head# 2024-09-14T22:55:06.467 INFO:teuthology.orchestra.run.smithi114.stdout:snapset 13=[]:{} 2024-09-14T22:55:06.467 INFO:teuthology.orchestra.run.smithi114.stdout:write_pg epoch 123 info 3.e( v 123'439 (0'0,123'439] local-lis/les=24/25 n=72 ec=24/24 lis/c=24/24 les/c/f=25/25/0 sis=24) 2024-09-14T22:55:06.469 INFO:teuthology.orchestra.run.smithi114.stdout:Import successful 2024-09-14T22:55:06.475 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:06.475 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:06.484 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:06.537 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:06.536+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:06.627 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:06.627 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:06.635 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:06.638 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:06.637+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:06.741 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:06.741 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:06.742 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:06.741+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:06.746 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:06.843 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:06.842+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:06.868 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:55:06.898 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:06.898 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:06.907 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:06.945 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:06.944+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:06.991 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /var/log/ceph/exp.3.e.4 2024-09-14T22:55:07.047 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:07.046+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:07.054 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:07.054 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:07.063 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:07.102 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:55:07.148 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:07.147+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:07.171 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:07.171 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:07.181 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:07.230 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:07.231 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:07.248 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:07.248 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:07.247+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:07.264 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:07.264 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:07.280 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:07.280 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:07.280 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:07.287 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:07.288 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:07.288 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:07.297 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:07.298 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:07.298 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:07.314 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:07.314 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:07.330 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:07.331 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:07.334 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:07.334 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:07.349 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:07.347+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:07.403 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:07.403 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:07.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:07.403 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:07.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.403 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.403 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.551200756, 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:07.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:55:07.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.55026316599999, 2024-09-14T22:55:07.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:55:07.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.550171803, 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.413 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.417 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.549526918, 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.418 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:55:07.422 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.54903953900001, 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.423 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:07.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.548152264, 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:55:07.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.54755984000001, 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:55:07.432 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:55:07.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.436 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.546827024, 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.437 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:55:07.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.441 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.546727875, 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:55:07.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.446 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.54640941700001, 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:55:07.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:55:07.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.54589992299999, 2024-09-14T22:55:07.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:55:07.455 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.545842064, 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.456 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:07.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.545049441, 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:55:07.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.544374002, 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:55:07.465 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:07.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:07.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:55:07.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:55:07.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.544240692, 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:55:07.472 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:07.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:55:07.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.54362376900001, 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:07.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:55:07.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:55:07.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.483 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.543556912, 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.484 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.54310610100001, 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:55:07.489 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:07.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.542985868, 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:55:07.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 103.541667741, 2024-09-14T22:55:07.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:55:07.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:55:07.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:55:07.505 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.506 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.506 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.506 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.506 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.506 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:07.507 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:07.508 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:07.506+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:07.608 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:07.607+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:07.611 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:07.611 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:07.611 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:07.611 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:07.620 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:07.709 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:07.708+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:07.764 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:07.764 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:07.764 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:07.764 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:07.764 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:07.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:07.813 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:07.812+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:07.911 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:07.909+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:07.928 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:07.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.929416810000006, 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.929 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:07.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:55:07.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:55:07.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.929397772000002, 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:55:07.934 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:55:07.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.938 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.929361815999997, 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:55:07.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:55:07.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:07.943 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.929340695999997, 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:55:07.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.928133782000003, 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.948 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:07.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.952 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.926304334999998, 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:55:07.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:55:07.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.925750047999998, 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.957 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:55:07.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.923765329999995, 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:55:07.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.923413010000004, 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.966 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:55:07.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.923129567000004, 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.971 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:55:07.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.922841469999995, 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.975 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:55:07.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:55:07.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.979 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.922466642000003, 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.980 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:55:07.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:55:07.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.922145826999994, 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:55:07.984 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:55:07.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.988 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.908460591999997, 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:55:07.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:55:07.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.908125783000003, 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:55:07.993 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:55:07.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:55:07.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:07.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.907795621000005, 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.998 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:55:07.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.907574835000005, 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.002 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.003 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.907305534000002, 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:55:08.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.907118437999998, 2024-09-14T22:55:08.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:55:08.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 90.906800278999995, 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:55:08.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.019 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.020 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.020 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.020 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:08.021 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:08.022 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:08.020+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:08.122 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:08.121+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:08.165 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:08.165 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:08.166 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:08.166 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:08.174 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:08.258 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:08.257+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:08.318 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:08.318 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:08.318 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:08.318 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:08.318 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:08.324 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:08.323+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:08.326 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:08.424 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:08.423+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:08.471 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 101.26082835, 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.472 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:55:08.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.476 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 88.137166575999998, 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:08.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:55:08.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:55:08.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:55:08.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 79.205399104999998, 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.482 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:08.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.295452863000001, 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.493 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.346792219999998, 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.494 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:55:08.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:55:08.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:55:08.499 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.890715342999997, 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.500 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:55:08.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.504 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.890688021000003, 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.505 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.509 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.890276530000001, 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:55:08.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.513 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.888617263, 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:08.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:55:08.517 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.884585839000003, 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:55:08.518 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.519 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.520 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.521 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:55:08.522 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194089 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.883998173000002, 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005182201, 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.523 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194089 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:08.524 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585353+0000", 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8899999999999998e-07 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585357+0000", 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8470000000000003e-06 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.525 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585369+0000", 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1992e-05 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585377+0000", 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6090000000000003e-06 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590237+0000", 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048596910000000002 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.526 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590352+0000", 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000115387 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590534+0000", 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018168599999999999 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.527 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194119 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 270336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.882762812999999, 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051579030000000001, 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194119 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.528 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.529 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586587+0000", 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8200000000000001e-07 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586590+0000", 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2699999999999999e-06 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586596+0000", 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.212e-06 2024-09-14T22:55:08.530 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586601+0000", 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8289999999999998e-06 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591330+0000", 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047292439999999996 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.531 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591418+0000", 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8448000000000004e-05 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591745+0000", 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00032651799999999998 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.882742362999998, 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:55:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.536 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.882722688999998, 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.537 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.538 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.539 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:55:08.540 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.882684034999997, 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.542 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.543 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.544 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:55:08.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 41.882664155000001, 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:55:08.546 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.548 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.549 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:55:08.550 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.509492962000003, 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.551 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.552 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.553 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.554 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:55:08.555 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.556 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 37.508491667999998, 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.557 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:08.558 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.559 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.560 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:55:08.561 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.562 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 36.202435477000002, 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:55:08.563 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:55:08.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:55:08.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.567 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:55:08.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:211119 3.4 3:26977db1:::rbd_data.11354571ccdf.0000000000000332:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.5857160369999992, 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0055643710000000002, 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.570 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 211119 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883635+0000", 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.032e-06 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883640+0000", 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1649999999999997e-06 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883653+0000", 2024-09-14T22:55:08.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3241000000000001e-05 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883661+0000", 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7389999999999999e-06 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883723+0000", 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2403000000000005e-05 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883845+0000", 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012148 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.884653+0000", 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00080859200000000001 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.886346+0000", 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001692268 2024-09-14T22:55:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889139+0000", 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027930920000000001 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889171+0000", 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2595999999999998e-05 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889198+0000", 2024-09-14T22:55:08.576 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6763000000000001e-05 2024-09-14T22:55:08.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.576 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.576 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.576 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:08.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:08.578 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:08.576+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:08.678 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:08.677+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:08.722 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:08.722 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:08.723 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:08.723 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:08.731 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:08.778 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:08.777+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:08.835 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:08.835 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:08.835 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:08.835 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:08.835 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:08.844 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:08.886 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:55:08.887 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:08.886+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:08.958 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.051272458, 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.959 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:55:08.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:08.964 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.05125192200001, 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.051221923, 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.969 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:55:08.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.05119077000001, 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:55:08.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:55:08.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:55:08.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:55:08.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:55:08.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.05116811400001, 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.979 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:55:08.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.051147613, 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.984 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:55:08.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:55:08.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:55:08.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.05112785199999, 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:55:08.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:55:08.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:08.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.051107067, 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:08.996 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:55:08.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:08.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:55:09.000 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.05107465499999, 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.001 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:55:09.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.051053256, 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.006 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.007 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:55:09.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.050991857, 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:55:09.010 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:55:09.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.050951944, 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.015 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:55:09.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.050922188, 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:55:09.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:55:09.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.050891953, 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:55:09.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:55:09.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.05086721399999, 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:55:09.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.031 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:55:09.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.034 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.05084378399999, 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:55:09.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.050748048, 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.039 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:55:09.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.043 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.050719126, 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.044 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.050695801, 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.049 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:55:09.050 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.051 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.052 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 83.917998057999995, 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:09.053 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.054 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.055 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:55:09.056 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:55:09.057 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:55:09.058 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.058 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.058 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.058 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:09.058 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:09.058 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:09.059 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:09.060 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:09.058+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:09.160 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:09.159+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:09.164 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:09.164 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:09.165 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:09.165 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:09.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:09.261 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:09.260+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:09.316 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:09.317 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:09.317 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:09.317 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:09.317 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:09.326 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:09.335 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:55:09.379 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:09.378+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:09.432 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:09.432 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:09.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:09.432 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:09.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.432 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:55:09.432 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 97.533660826000002, 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:55:09.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:55:09.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:55:09.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:55:09.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.439 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.907866706999997, 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.440 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.907778715000006, 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.445 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:55:09.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.907544798000004, 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:55:09.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:55:09.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:55:09.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:55:09.454 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.907363903000004, 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.455 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:09.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:55:09.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.907064399999996, 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.460 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:55:09.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.906718190000007, 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.465 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:55:09.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.469 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.906354461999996, 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:55:09.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:55:09.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.905764844999993, 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:55:09.474 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.478 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.905562607999997, 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.479 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:55:09.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.483 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.905372796999998, 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:55:09.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.905060793999994, 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.488 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:55:09.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.904818306999999, 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.493 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:55:09.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.904659344999999, 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:55:09.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.904025075000007, 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:55:09.502 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:55:09.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:55:09.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.903810965000005, 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.507 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.903674495000004, 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.512 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:55:09.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.903422015000004, 2024-09-14T22:55:09.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:55:09.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 41.804523691, 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:09.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:55:09.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:55:09.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 29.309037062000002, 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:09.528 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:55:09.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:55:09.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:55:09.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:09.534 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:09.535 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:09.536 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:09.536 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:09.536 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:09.537 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:09.537 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:09.538 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:09.538 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:09.538 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:09.538 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:09.539 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:09.539 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:09.539 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:09.534+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:09.540 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:09.540 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:09.554 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:09.554 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:09.557 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:09.557 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:09.637 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:09.635+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:09.643 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:09.643 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:09.651 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:09.737 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:09.736+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:09.795 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:09.795 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:09.803 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:09.839 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:09.837+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:09.882 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:55:09.911 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:09.912 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:09.921 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:09.939 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:09.938+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:10.029 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:10.029 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:10.039 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:10.040 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:10.039+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:10.140 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:10.140+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:10.141 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:10.141 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:10.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:10.241 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:10.240+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:10.253 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:10.253 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:10.261 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:10.342 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:10.341+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:10.366 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:10.366 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:10.374 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:10.443 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:10.442+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:10.476 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:10.476 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:10.485 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:10.569 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:10.568+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:10.628 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:10.629 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:10.638 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:10.645 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:10.644+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:10.746 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:10.745+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:10.781 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:10.781 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:10.789 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:10.847 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:10.845+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:10.897 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:10.897 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:10.906 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:10.948 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:10.947+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:11.049 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:11.047+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:11.056 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:11.056 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:11.065 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:11.149 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:11.148+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:11.168 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:11.168 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:11.177 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:11.255 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:11.254+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:11.321 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:11.321 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:11.330 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:11.374 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:11.373+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:11.433 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:11.434 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:11.445 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:11.452 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:11.451+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:11.553 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:11.552+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:11.558 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:55:11.590 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:11.590 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:11.599 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:11.680 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:11.681 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:11.681 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:11.679+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:11.698 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:11.698 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:11.714 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:11.714 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:11.730 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:11.730 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:11.731 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:11.746 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:11.747 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:11.747 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:11.747 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:11.747 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:11.748 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:11.748 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:11.754 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:11.753+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:11.756 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:11.765 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:11.765 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:11.782 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:11.782 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:11.786 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:11.786 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:11.856 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:11.854+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:11.870 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:11.870 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:11.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:11.870 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:11.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963878663, 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.871 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:55:11.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:11.875 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96385812699999, 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:55:11.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963828128, 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.880 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:55:11.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:55:11.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:55:11.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96379697499999, 2024-09-14T22:55:11.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:55:11.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:55:11.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963774319, 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.890 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:55:11.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.894 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963753818, 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.895 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:55:11.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.899 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963734057, 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:55:11.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:55:11.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96371327200001, 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.904 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:55:11.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:55:11.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.908 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96368086, 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:11.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:55:11.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.913 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96365946100001, 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:55:11.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:55:11.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.918 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963598062, 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:55:11.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96355814899999, 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.923 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:55:11.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963528393, 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.928 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:11.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96349815799999, 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:55:11.932 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:55:11.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963473419, 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:55:11.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:55:11.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.963449989, 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:55:11.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.946 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96335425300001, 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:55:11.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:55:11.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96332533099999, 2024-09-14T22:55:11.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:55:11.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.96330200600001, 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:11.956 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:55:11.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.960 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 86.830604262999998, 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:11.961 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:11.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:11.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.965 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:11.966 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:11.966 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:11.966 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:55:11.968 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:11.968 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:11.966+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:12.065 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T22:55:12.065 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:55:12.066 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T22:55:12.066 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T22:55:12.068 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:12.066+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:12.073 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:12.073 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:12.073 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:12.073 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:12.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:12.169 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:12.168+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:12.201 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:12.201 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:12.201 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:12.202 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:12.202 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:12.202 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:12.202 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:12.202 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:12.202 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:12.202 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:12.202 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:12.203 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:12.204 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:12.205 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:12.205 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-14T22:55:12.224 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:12.224 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:12.224 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:12.232 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:12.276 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:12.275+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:12.336 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:12.336 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 100.438199193, 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:12.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:55:12.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:55:12.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:55:12.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:55:12.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.345 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.812405073999997, 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.346 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:55:12.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.812317082000007, 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.351 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:55:12.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:55:12.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:12.356 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.812083165000004, 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:55:12.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.811902270000004, 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.362 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:55:12.366 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.811602766999997, 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.367 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:55:12.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:12.372 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.811256556999993, 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:55:12.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.810892828999997, 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.378 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:55:12.382 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.810303211999994, 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.383 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:55:12.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.810100974999997, 2024-09-14T22:55:12.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:55:12.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:55:12.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.809911163999999, 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.394 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:55:12.398 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.809599160999994, 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.399 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:55:12.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:55:12.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.809356674, 2024-09-14T22:55:12.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:55:12.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.409 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.809197712, 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.410 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:55:12.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.414 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.808563441999993, 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.415 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:55:12.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.808349332000006, 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:55:12.420 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:55:12.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:55:12.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.425 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.808212862000005, 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:55:12.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.807960382000005, 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.431 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:55:12.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:55:12.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.709062058000001, 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:55:12.436 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:55:12.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:12.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:55:12.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:55:12.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.213575429000002, 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.444 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:12.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:55:12.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:55:12.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:55:12.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:55:12.451 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.451 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.451 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.451 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.451 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.451 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:12.453 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:12.454 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:12.454 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:12.454 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:12.454 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:12.454 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:12.455 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:12.455 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:12.456 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T22:55:12.456 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:12.451+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:12.553 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:12.552+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:12.597 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:12.597 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:12.597 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:12.606 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:12.625 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:12.626 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:12.628 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:12.628 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:12.629 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-14T22:55:12.669 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:12.668+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:12.707 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:12.707 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:12.707 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:12.708 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:12.708 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:12.716 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:12.754 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:12.753+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:12.800 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:12.800 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:12.800 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:12.800 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:12.800 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:12.801 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:12.801 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:12.801 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:12.801 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:12.801 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:12.801 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:12.801 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:12.803 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:12.803 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:12.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T22:55:12.822 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:12.822 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:12.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:12.822 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:12.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.822 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.73844741400001, 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:55:12.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.738229518, 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.827 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:55:12.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.735545872, 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:55:12.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:55:12.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:12.836 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.735387457, 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:12.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:55:12.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.735360191, 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.841 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:55:12.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.845 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.73515304999999, 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.846 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:55:12.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.735118786, 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:55:12.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:55:12.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.855 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.73348674, 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:55:12.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.733462128, 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.860 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:55:12.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.73332784199999, 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.865 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.73217594899999, 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.869 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:55:12.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:55:12.873 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.874 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.876 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.876 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.73215204500001, 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.880 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.732118525, 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.881 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:55:12.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.73196387900001, 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:55:12.885 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:55:12.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.731922705, 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:55:12.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:55:12.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.731894545, 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.895 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:55:12.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:12.899 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 110.731705154, 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 108.815615327, 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:55:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:55:12.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:55:12.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 108.815595496, 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.913 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.615889170999999, 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:55:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:55:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:12.919 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:12.921 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:55:12.922 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:12.922 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:12.919+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:13.021 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:13.020+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:13.029 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:13.029 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:13.030 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:13.030 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:13.038 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:13.039 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:13.042 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:13.042 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:13.042 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T22:55:13.122 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:13.121+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:13.174 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:13.175 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:13.177 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:13.177 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:13.177 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-14T22:55:13.184 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:13.184 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:13.184 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:13.184 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:13.184 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:13.192 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:13.236 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:13.235+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:13.296 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:13.296 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:13.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:13.296 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:13.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.296 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:55:13.296 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 140.78624538099999, 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:55:13.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:55:13.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.303 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.921587525999996, 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:55:13.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.921481589999999, 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.308 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:55:13.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.312 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.921406888999996, 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:55:13.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.317 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.921321407999997, 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:55:13.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.921175022, 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.322 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:55:13.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.921125021999998, 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:55:13.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:55:13.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.331 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.920961857999998, 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:55:13.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:55:13.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.920923548000005, 2024-09-14T22:55:13.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:55:13.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.920789878999997, 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.341 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:55:13.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.345 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.920709312, 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.346 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:55:13.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.350 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.920612375000005, 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:55:13.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:55:13.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.355 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.920560820000006, 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:55:13.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.917436190999993, 2024-09-14T22:55:13.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:13.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:55:13.364 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.917304578, 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.365 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:13.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 96.917018166000005, 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.370 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:55:13.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:55:13.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.858981912000004, 2024-09-14T22:55:13.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:13.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:55:13.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.858942084000006, 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.381 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:13.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:55:13.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:13.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.386 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.858926069000006, 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.387 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:55:13.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:55:13.392 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.809797713999998, 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:13.393 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:13.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:55:13.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:55:13.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:13.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:55:13.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:55:13.399 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.399 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.399 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.399 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:13.399 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:13.399 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:13.401 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:13.401 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:13.401 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:13.401 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:13.402 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:13.403 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:13.403 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-14T22:55:13.403 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:13.399+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:13.501 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:13.500+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:13.550 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:13.551 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:13.551 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:13.551 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:13.560 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:13.578 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:13.578 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:13.578 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:13.579 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:13.581 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:13.582 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:13.622 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:13.621+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:13.681 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:13.681 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:13.681 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:13.681 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:13.681 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:13.693 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:13.702 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:13.701+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:13.787 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:55:13.831 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:13.830+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:13.840 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.988768609, 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.841 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.843 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:55:13.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.987831019, 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.846 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:55:13.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.987739656, 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:13.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:55:13.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.987094771, 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.855 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.859 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.986607392, 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:13.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:55:13.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.985720117, 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:55:13.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:55:13.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.869 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.985127693, 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.870 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:55:13.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.874 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.984394877, 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:55:13.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:55:13.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:13.879 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.98429572800001, 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:13.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:55:13.883 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.98397727, 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.884 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:13.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:55:13.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.888 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.983467776, 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.889 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:55:13.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:13.893 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.983409917, 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:55:13.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.98261729399999, 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.898 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:55:13.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:55:13.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.902 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.981941855, 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:55:13.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:55:13.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.907 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.98180854500001, 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:55:13.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:55:13.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.912 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.981191622, 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:55:13.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.981124765, 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:55:13.917 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:55:13.921 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.980673954, 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.922 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.923 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.923 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:55:13.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.923 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.923 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:55:13.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.98055372100001, 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.927 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:13.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:55:13.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.979235594, 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.932 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:55:13.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:13.936 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:13.938 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:13.938 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:13.939 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:13.939 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:13.939 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:13.939 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:13.936+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:13.953 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:13.954 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:13.971 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:13.971 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:13.971 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:13.993 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:14.011 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:14.011 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:14.029 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:14.029 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:14.033 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:14.033 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:14.067 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:14.067 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:14.067 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:14.065+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:14.076 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:14.159 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:14.158+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:14.219 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:14.219 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:14.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:14.241 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:14.240+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:14.342 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:14.341+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:14.370 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:14.370 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:14.379 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:14.443 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:14.442+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:14.482 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:14.482 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:14.491 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:14.545 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:14.544+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:14.634 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:14.634 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:14.643 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:14.645 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:14.644+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:14.746 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:14.746 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:14.747 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:14.745+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:14.755 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:14.847 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:14.846+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:14.899 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:14.900 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:14.908 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:14.923 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:55:14.948 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:14.947+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:15.026 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:15.026 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:15.035 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:15.049 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:15.047+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:15.149 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:15.148+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:15.154 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:15.154 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:15.164 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:15.250 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:15.249+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:15.271 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:15.271 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:15.280 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:15.351 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:15.350+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:15.424 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:15.424 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:15.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:15.476 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:15.475+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:15.535 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:15.535 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:15.544 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:15.553 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:15.552+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:15.654 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:15.653+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:15.687 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:15.687 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:15.695 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:15.755 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:15.753+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:15.797 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:15.797 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:15.806 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:15.855 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:15.854+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:15.909 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:15.909 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:15.918 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:15.922 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:55:15.962 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:15.961+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:16.029 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:16.030 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:16.034 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:55:16.038 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:16.078 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:16.077+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:16.153 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:16.154 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:16.158 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:16.157+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:16.170 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:16.170 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:16.182 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:16.182 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:16.186 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:16.187 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:16.191 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:16.202 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:16.203 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:16.203 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:16.219 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:16.219 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:16.235 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:16.252 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:16.252 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:16.255 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:16.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:16.275 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:16.274+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:16.342 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:16.342 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:16.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:16.342 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:16.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.342 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 143.82672289300001, 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:55:16.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:55:16.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:16.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.962065038000006, 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.353 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:55:16.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:16.358 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961959101999994, 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.359 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:55:16.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:55:16.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:55:16.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961884401000006, 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.365 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.366 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.366 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.366 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:55:16.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.366 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:16.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:16.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:55:16.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:16.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:55:16.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:55:16.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:55:16.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:55:16.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:55:16.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:16.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:55:16.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:55:16.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:55:16.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:55:16.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:55:16.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:55:16.374 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.374 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.374 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.374 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.375 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:16.375 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961798920000007, 2024-09-14T22:55:16.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:55:16.375 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.375 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.375 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.375 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.376 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.376 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.376 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:55:16.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.376 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:16.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:16.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:16.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:55:16.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:55:16.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:55:16.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:55:16.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:55:16.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:55:16.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:55:16.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:55:16.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:55:16.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:55:16.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:55:16.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:55:16.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:55:16.385 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.385 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.385 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.386 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.386 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:16.386 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961652533999995, 2024-09-14T22:55:16.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:55:16.386 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.387 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.387 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.387 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:55:16.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.388 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:16.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:16.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:55:16.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:55:16.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:55:16.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:55:16.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:55:16.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:55:16.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:55:16.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:55:16.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:55:16.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:55:16.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:55:16.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:55:16.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:55:16.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:55:16.395 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.395 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.395 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.396 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.396 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:16.396 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961602533999994, 2024-09-14T22:55:16.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:55:16.396 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.396 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.396 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:55:16.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:55:16.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:55:16.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:55:16.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:55:16.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:16.401 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961439369999994, 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:16.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.96140106, 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:55:16.406 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961267391000007, 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:55:16.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:55:16.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:55:16.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:55:16.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:55:16.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:55:16.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:55:16.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:55:16.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:55:16.417 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.417 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.417 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.418 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.418 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:16.418 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961186823999995, 2024-09-14T22:55:16.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:55:16.418 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.418 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.418 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.419 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.419 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.419 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.419 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:55:16.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.419 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:16.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:16.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:16.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:55:16.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.421 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:55:16.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:55:16.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:55:16.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:55:16.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:55:16.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:55:16.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:55:16.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:55:16.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:55:16.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:55:16.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:55:16.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:55:16.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.425 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.426 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.426 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:16.426 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961089887, 2024-09-14T22:55:16.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:55:16.426 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.426 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.427 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.427 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.427 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.427 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.427 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:55:16.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.427 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:16.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:16.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:16.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:55:16.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:16.432 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.961038332000001, 2024-09-14T22:55:16.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:55:16.433 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.433 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.433 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.433 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.433 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.433 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.433 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:55:16.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:55:16.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:55:16.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.437 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.957913703000003, 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.438 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.439 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:55:16.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.439 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:16.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:55:16.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.443 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.957782089999995, 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:55:16.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:55:16.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:55:16.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:55:16.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 99.957495678000001, 2024-09-14T22:55:16.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:16.455 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.899459424, 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.456 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:16.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:55:16.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:55:16.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:16.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:55:16.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:16.464 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.899419596000001, 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:55:16.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:55:16.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:16.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:55:16.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:55:16.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:16.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:55:16.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:55:16.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:16.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:55:16.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:55:16.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:16.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:55:16.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:55:16.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:55:16.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:55:16.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.472 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.472 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.473 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:16.473 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:16.473 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.899403581000001, 2024-09-14T22:55:16.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:55:16.473 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.473 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.474 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:16.474 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.474 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.474 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.474 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:55:16.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.474 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:16.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:16.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:16.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:55:16.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:55:16.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:55:16.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:55:16.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:55:16.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:55:16.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:55:16.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:55:16.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:55:16.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:55:16.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:55:16.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:16.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:55:16.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:55:16.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:16.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:55:16.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:55:16.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:16.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:55:16.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:55:16.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:16.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:55:16.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:55:16.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:55:16.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:55:16.484 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.484 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.484 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.485 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:55:16.485 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:16.485 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 71.850275225999994, 2024-09-14T22:55:16.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:55:16.486 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:16.486 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:16.486 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:16.486 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:16.486 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:16.486 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.487 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:55:16.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.487 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:16.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:16.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:16.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:16.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:16.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:16.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:16.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:55:16.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:55:16.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:16.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:55:16.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:55:16.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:16.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:55:16.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:55:16.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:55:16.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:55:16.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:16.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:55:16.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:55:16.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:16.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:55:16.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:55:16.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:16.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:55:16.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:55:16.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:16.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:55:16.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:55:16.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:16.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:55:16.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:55:16.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:16.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:55:16.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:55:16.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:16.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:16.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:16.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:55:16.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:55:16.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.497 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.497 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.497 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:16.497 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:16.497 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:16.500 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:16.498+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:16.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:16.600 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:16.599+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:16.609 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:16.609 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:16.609 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:16.609 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:16.617 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:16.701 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:16.700+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:16.762 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:16.763 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:16.763 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:16.763 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:16.763 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:16.771 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:16.815 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:16.814+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:16.877 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:16.877 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:16.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:16.877 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:16.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.878 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.878 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:16.878 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.02419730299999, 2024-09-14T22:55:16.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:55:16.878 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.878 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.879 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.879 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:55:16.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.879 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:55:16.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:55:16.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:55:16.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:55:16.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:55:16.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.885 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.885 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:16.885 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.023259713, 2024-09-14T22:55:16.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:55:16.885 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.885 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.885 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.886 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.886 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.886 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:55:16.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.886 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:55:16.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:16.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.891 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.02316835000001, 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.892 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:16.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:55:16.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:55:16.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:55:16.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:55:16.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:55:16.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:55:16.897 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.897 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.897 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.02252346500001, 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.898 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:55:16.902 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.022036086, 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:16.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.907 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.021148811, 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.908 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:55:16.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.020556387, 2024-09-14T22:55:16.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:55:16.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.918 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.019823571, 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.919 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:55:16.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.019724422, 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:55:16.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:16.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:55:16.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:55:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.930 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.019405964, 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:16.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:16.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:55:16.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:55:16.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.01889647, 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:55:16.936 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:16.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:16.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:55:16.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:55:16.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.941 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:55:16.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:55:16.942 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.942 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.942 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.942 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.942 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:16.943 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.01883861100001, 2024-09-14T22:55:16.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:55:16.943 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.943 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.943 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.943 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.943 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:55:16.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:55:16.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.948 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.018045988, 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.949 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:55:16.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:55:16.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.01737054900001, 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.954 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:55:16.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:55:16.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.017237239, 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.959 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.963 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.016620316, 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:55:16.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.01655345899999, 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:55:16.968 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:55:16.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:55:16.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:55:16.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:55:16.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:55:16.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.016102648, 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.974 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:16.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:55:16.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:55:16.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:55:16.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:55:16.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:55:16.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:55:16.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.015982415, 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:55:16.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:55:16.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.01466428800001, 2024-09-14T22:55:16.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:55:16.986 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:16.986 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:16.986 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:16.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:16.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:16.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:16.986 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:55:16.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:16.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:16.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:16.990 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:16.992 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:16.992 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:16.990+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:17.092 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:17.091+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:17.136 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:17.137 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:17.137 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:17.137 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:17.145 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:17.193 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:17.191+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:17.249 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:17.249 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:17.249 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:17.249 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:17.249 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:17.258 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:17.293 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:17.292+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:17.394 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:17.393+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:17.404 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:17.404 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:17.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:17.404 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.405824881, 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:55:17.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:55:17.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:55:17.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:55:17.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:55:17.409 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.405805843, 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.410 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:55:17.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.40576988700001, 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:55:17.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.40574876700001, 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:55:17.419 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:55:17.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.404541853, 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.424 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.428 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.40271240600001, 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.429 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:55:17.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.40215811900001, 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.433 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.437 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.400173401, 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:55:17.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.399821081, 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.442 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:17.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:55:17.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.399537638, 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.447 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:55:17.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:17.451 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.399249541, 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:55:17.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.398874713, 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.456 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:55:17.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:55:17.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.460 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.398553898, 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:55:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:55:17.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.465 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.38486866300001, 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:55:17.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.384533854, 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.470 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:55:17.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.474 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.384203692, 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:55:17.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:55:17.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.383982906, 2024-09-14T22:55:17.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:17.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:55:17.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.383713605, 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.484 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:17.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.38352650900001, 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:55:17.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.492 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.38320835, 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.493 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:55:17.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:55:17.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:55:17.497 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.497 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.497 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.497 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.497 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.497 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:17.499 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:17.499 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:17.497+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:17.599 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:17.598+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:17.642 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:17.643 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:17.643 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:17.643 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:17.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:17.735 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:17.734+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:17.796 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:17.796 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:17.796 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:17.801 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:17.800+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:17.805 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:17.902 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:17.901+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:17.942 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 110.742768631, 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.619106857000006, 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.958 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:55:17.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:55:17.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:17.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:17.962 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 88.687339386000005, 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.966 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.777393144000001, 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:55:17.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.971 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.828732500999998, 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.972 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:55:17.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.977 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.372655623999997, 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:55:17.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.981 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.372628302000003, 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.982 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.372216811000001, 2024-09-14T22:55:17.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:55:17.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.370557544, 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.990 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.366526120000003, 2024-09-14T22:55:17.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.996 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.997 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.998 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194089 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.365938454000002, 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005182201, 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194089 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:17.999 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585353+0000", 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8899999999999998e-07 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.000 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585357+0000", 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8470000000000003e-06 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585369+0000", 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1992e-05 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585377+0000", 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6090000000000003e-06 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.001 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590237+0000", 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048596910000000002 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590352+0000", 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000115387 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590534+0000", 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018168599999999999 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.002 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.364682643999998, 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.003 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.004 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:55:18.006 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.364662969999998, 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:55:18.007 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.008 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.009 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.010 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.364624315999997, 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:55:18.011 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.012 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.013 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.014 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:18.015 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 51.364604436, 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:55:18.017 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:55:18.018 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.019 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 46.991433243000003, 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.020 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:55:18.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:55:18.022 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.023 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.024 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:18.025 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 46.990431948999998, 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:18.026 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:55:18.027 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.028 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.029 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.030 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 45.684375758000002, 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.031 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.032 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.033 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.034 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:18.035 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.036 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:211119 3.4 3:26977db1:::rbd_data.11354571ccdf.0000000000000332:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 18.067656318000001, 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0055643710000000002, 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.037 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 211119 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:18.038 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883635+0000", 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.032e-06 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883640+0000", 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1649999999999997e-06 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.039 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883653+0000", 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3241000000000001e-05 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883661+0000", 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7389999999999999e-06 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883723+0000", 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2403000000000005e-05 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.040 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883845+0000", 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012148 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.884653+0000", 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00080859200000000001 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.886346+0000", 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001692268 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.041 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889139+0000", 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027930920000000001 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889171+0000", 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2595999999999998e-05 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889198+0000", 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6763000000000001e-05 2024-09-14T22:55:18.042 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:213874 3.3 3:c31b501c:::rbd_header.1135a4365f9c:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 1.0125780600000001, 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0068492850000000001, 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.043 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 213874 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.044 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938714+0000", 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1059999999999998e-06 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938723+0000", 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0180000000000007e-06 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938745+0000", 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1985e-05 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.045 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938755+0000", 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0051000000000001e-05 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938843+0000", 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7671999999999997e-05 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938849+0000", 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6080000000000001e-06 2024-09-14T22:55:18.046 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941261+0000", 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0024116459999999999 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941361+0000", 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9846999999999995e-05 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941720+0000", 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00035947399999999998 2024-09-14T22:55:18.047 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945393+0000", 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036722679999999998 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945439+0000", 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6186000000000001e-05 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945507+0000", 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7947999999999994e-05 2024-09-14T22:55:18.048 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945561+0000", 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4475999999999998e-05 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.049 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:18.051 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:18.051 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:18.049+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:18.151 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:18.150+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:18.197 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:18.197 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:18.197 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:18.197 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:18.206 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:18.252 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:18.251+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:18.257 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:55:18.311 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:18.311 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:18.311 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:18.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:18.363 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:18.362+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:18.379 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:18.380 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:18.396 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:18.413 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:18.413 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:18.427 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:18.427 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:18.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:18.427 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520971401, 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:55:18.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:18.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:55:18.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520950865, 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.432 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.433 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.436 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520920866, 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.437 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:55:18.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520889713, 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.441 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:18.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.445 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520867057, 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.446 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:55:18.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:55:18.449 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520846556, 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.450 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.454 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520826795, 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:55:18.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.457 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.52080601, 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.459 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:55:18.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.52077359800001, 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:55:18.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520752199, 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.468 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:55:18.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.5206908, 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.473 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:55:18.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520650887, 2024-09-14T22:55:18.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:18.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:55:18.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520621131, 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.482 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.483 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:55:18.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520590896, 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:55:18.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:55:18.489 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.490 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520566157, 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.491 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.492 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:55:18.493 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:55:18.494 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.495 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.52054272700001, 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.496 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.497 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.498 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.499 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520446991, 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.500 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.501 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.502 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:55:18.503 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520418069, 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.504 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.505 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.506 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.507 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.520394744, 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.508 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:18.509 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:55:18.510 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:18.511 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:55:18.512 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 93.387697001000006, 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.513 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.514 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.515 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:18.516 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:18.517 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:18.518 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:18.518 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:18.519 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:18.519 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:18.520 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:18.520 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:18.520 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:18.521 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:18.521 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:18.522 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:18.522 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:18.522 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:18.522 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:18.517+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:18.583 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T22:55:18.583 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:55:18.583 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T22:55:18.583 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-14T22:55:18.619 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:18.617+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:18.626 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:18.627 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:18.635 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:18.719 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:18.717+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:18.778 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:18.779 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:18.787 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:18.821 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:18.818+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:18.872 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-14T22:55:18.869+0000 7f721e037640 -1 mon.a@0(leader).osd e144 definitely_dead 0 2024-09-14T22:55:18.891 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:18.892 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:18.902 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:18.921 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:18.919+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:18.945 INFO:teuthology.orchestra.run.smithi076.stderr:marked out osd.1. 2024-09-14T22:55:18.950 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:55:19.010 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:19.010 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:19.018 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:19.022 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:19.020+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:19.121 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:19.122 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:19.122 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:19.121+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:19.131 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:19.224 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:19.222+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:19.274 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:19.275 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:19.283 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:19.325 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:19.323+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:19.385 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:19.385 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:19.395 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:19.439 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:19.438+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:19.497 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:19.497 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:19.506 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:19.547 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:19.545+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:19.627 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:19.625+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:19.649 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:19.649 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:19.658 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:19.728 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:19.726+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:19.766 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:19.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:19.829 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:19.828+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:19.917 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:19.917 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:19.925 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:19.929 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:19.929+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:20.031 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:20.028+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:20.031 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:20.032 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:20.040 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:20.131 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:20.130+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:20.184 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:20.184 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:20.192 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:20.232 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:20.231+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:20.295 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:20.304 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:20.332 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:20.332+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:20.406 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:20.407 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:20.415 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:20.434 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:20.432+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:20.518 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:20.518 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:20.523 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:55:20.527 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:20.535 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:20.533+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:20.636 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:20.633+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:20.646 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:20.647 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:20.664 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:20.664 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:20.673 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:20.673 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:20.680 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:20.680 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:20.682 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:20.696 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:20.697 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:20.714 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:20.714 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:20.731 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:20.731 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:20.737 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:20.735+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:20.748 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:20.749 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:20.752 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:20.753 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 113.59178382100001, 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:55:20.803 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.804 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.804 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.804 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:55:20.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:55:20.808 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 100.46812204699999, 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:55:20.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:55:20.813 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 91.536354575999994, 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:55:20.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.817 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.818 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 66.626408334000004, 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.819 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.820 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:55:20.821 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.822 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:55:20.823 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 57.677747691, 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.824 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:55:20.825 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.826 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.827 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:20.828 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.829 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.221670813999999, 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.830 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.831 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:55:20.832 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.833 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.834 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.221643491999998, 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.835 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:55:20.836 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.837 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.838 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.221232000999997, 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.839 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.840 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:55:20.841 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:55:20.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:55:20.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.219572734000003, 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:55:20.844 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:55:20.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.848 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.215541309999999, 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:55:20.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:55:20.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:55:20.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194089 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.214953643999998, 2024-09-14T22:55:20.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005182201, 2024-09-14T22:55:20.855 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.855 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.855 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.855 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.855 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.855 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.855 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194089 2024-09-14T22:55:20.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585352+0000", 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585353+0000", 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8899999999999998e-07 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585357+0000", 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8470000000000003e-06 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585369+0000", 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1992e-05 2024-09-14T22:55:20.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585377+0000", 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6090000000000003e-06 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590237+0000", 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0048596910000000002 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590352+0000", 2024-09-14T22:55:20.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000115387 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590534+0000", 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018168599999999999 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.859 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.213697834000001, 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:55:20.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.213678160000001, 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.864 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:55:20.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.213639506, 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.869 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:55:20.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.873 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.213619626000003, 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:55:20.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.840448432999999, 2024-09-14T22:55:20.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:20.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:55:20.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 49.839447139000001, 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.885 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:55:20.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:55:20.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 48.533390947999997, 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.891 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:55:20.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:55:20.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.896 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:211119 3.4 3:26977db1:::rbd_data.11354571ccdf.0000000000000332:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 20.916671508, 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0055643710000000002, 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.897 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 211119 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883635+0000", 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.032e-06 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883640+0000", 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1649999999999997e-06 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883653+0000", 2024-09-14T22:55:20.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3241000000000001e-05 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883661+0000", 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7389999999999999e-06 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883723+0000", 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2403000000000005e-05 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883845+0000", 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012148 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.884653+0000", 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00080859200000000001 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.886346+0000", 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001692268 2024-09-14T22:55:20.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889139+0000", 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027930920000000001 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889171+0000", 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2595999999999998e-05 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889198+0000", 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6763000000000001e-05 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:213874 3.3 3:c31b501c:::rbd_header.1135a4365f9c:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 3.8615932499999999, 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0068492850000000001, 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:20.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 213874 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:20.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938714+0000", 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1059999999999998e-06 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938723+0000", 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0180000000000007e-06 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938745+0000", 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1985e-05 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938755+0000", 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0051000000000001e-05 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938843+0000", 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7671999999999997e-05 2024-09-14T22:55:20.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938849+0000", 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6080000000000001e-06 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941261+0000", 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0024116459999999999 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:20.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941361+0000", 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9846999999999995e-05 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941720+0000", 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00035947399999999998 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945393+0000", 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036722679999999998 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945439+0000", 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6186000000000001e-05 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945507+0000", 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7947999999999994e-05 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:20.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945561+0000", 2024-09-14T22:55:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4475999999999998e-05 2024-09-14T22:55:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:20.910 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:20.910 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:20.911 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:20.912 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:20.910+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:20.966 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:55:21.012 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:21.010+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:21.039 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:21.039 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:21.040 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:21.040 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:21.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:21.112 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:21.111+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:21.194 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:21.195 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:21.195 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:21.195 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:21.195 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:21.206 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:21.213 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:21.212+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:21.314 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:21.312+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:21.321 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.41529625699999, 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.322 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:55:21.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:55:21.326 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.415275721, 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.327 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:55:21.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.41524572199999, 2024-09-14T22:55:21.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:21.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.334 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.335 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.415214569, 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.336 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:55:21.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.340 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.415191913, 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:55:21.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:55:21.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.41517141200001, 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.345 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:55:21.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.415151651, 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:55:21.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.352 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:55:21.353 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.354 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.415130866, 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.355 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:21.356 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.357 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.358 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.415098454, 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:55:21.359 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.360 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:55:21.361 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.363 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.415077055, 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.364 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.365 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:55:21.366 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.367 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.368 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.41501565599999, 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.369 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:55:21.371 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.372 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.414975743, 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.373 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.374 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:21.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.376 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.377 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.414945987, 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:55:21.378 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.379 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:55:21.380 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.381 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.382 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.414915752, 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.383 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.384 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:55:21.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:55:21.386 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.414891013, 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.387 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:55:21.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:21.391 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.414867583, 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.392 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:55:21.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.396 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.414771847, 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.397 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:55:21.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.414742925, 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:55:21.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.405 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.4147196, 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:55:21.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:55:21.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 96.282021857000004, 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:55:21.410 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:21.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:21.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:55:21.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:21.415 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:21.417 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:21.417 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:21.415+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:21.518 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:21.516+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:21.521 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:21.521 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:21.521 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:21.521 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:21.529 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:21.619 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:21.617+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:21.672 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:21.672 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:21.672 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:21.672 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:21.672 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:21.680 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:21.721 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:21.719+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:21.784 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:21.784 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.886057963, 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.785 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:55:21.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:55:21.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:55:21.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.26026384399999, 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:55:21.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.796 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.260175852, 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:55:21.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:55:21.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.259941935, 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:55:21.801 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:55:21.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:55:21.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.25976104, 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:55:21.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.25946153700001, 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.810 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:55:21.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.259115327, 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.815 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:55:21.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.25875159899999, 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.819 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:55:21.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.823 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.258161982, 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:55:21.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.25795974499999, 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:21.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:55:21.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.832 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.257769934, 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.833 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:55:21.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.257457931, 2024-09-14T22:55:21.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:55:21.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:55:21.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.257215444, 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:55:21.843 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:55:21.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.257056482, 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:55:21.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:55:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.256422212, 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:55:21.852 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:55:21.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:55:21.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.256208102, 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:55:21.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.861 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.256071632, 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:55:21.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 104.255819152, 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.870 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.156920827999997, 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:55:21.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:55:21.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:21.878 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 41.661434198999999, 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:21.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:21.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:55:21.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:55:21.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:21.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:55:21.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:55:21.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:21.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:21.885 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:21.886 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:21.887 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:21.841+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:21.943 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:21.942+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:21.970 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:55:21.993 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:21.993 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:21.994 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:21.994 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:22.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:22.045 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:22.043+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:22.104 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:22.105 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:22.105 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:22.105 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:22.105 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:22.113 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:22.146 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:22.144+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:22.217 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:22.217 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.13336092599999, 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:55:22.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:55:22.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:55:22.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.13314303, 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:55:22.222 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:55:22.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:55:22.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.226 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.13045938400001, 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:55:22.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:55:22.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.130300969, 2024-09-14T22:55:22.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:22.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:55:22.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.130273703, 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.236 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:22.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.240 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.130066562, 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:55:22.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:55:22.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:55:22.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.130032298, 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.245 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.12840025200001, 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.249 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:55:22.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:55:22.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:55:22.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.12837564, 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.254 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:22.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.258 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.128241354, 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.259 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:55:22.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.127089461, 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.264 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.12706555699999, 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:55:22.268 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:55:22.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.12703203700001, 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:55:22.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.277 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.12687739099999, 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:55:22.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.282 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.126836217, 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:55:22.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:22.287 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.12680805700001, 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:55:22.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:55:22.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.126618666, 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.292 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:55:22.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.296 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.21052883900001, 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.297 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:55:22.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.210509008, 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:55:22.301 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:55:22.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.010802683000001, 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.306 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:55:22.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:55:22.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:55:22.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.312 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.312 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.312 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:22.313 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:22.314 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:22.312+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:22.415 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:22.413+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:22.416 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:22.416 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:22.417 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:22.417 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:22.425 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:22.515 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:22.514+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:22.570 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:22.570 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:22.570 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:22.570 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:22.570 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:22.579 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:22.617 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:22.615+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:22.717 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:22.715+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:22.722 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:22.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 150.21276861600001, 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.723 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:55:22.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:22.729 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.348110761, 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:22.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:55:22.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.733 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.348004825, 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.734 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:55:22.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347930124, 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.738 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.742 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347844643, 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:55:22.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347698257, 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.747 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:22.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.751 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347648257, 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:55:22.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347485093, 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.756 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:55:22.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.760 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347446783, 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.761 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:55:22.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347313114, 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:55:22.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347232547, 2024-09-14T22:55:22.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:55:22.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.34713561, 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:22.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:55:22.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.347084055, 2024-09-14T22:55:22.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:55:22.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.343959426, 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:55:22.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:55:22.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.789 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.343827813, 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:55:22.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 106.343541401, 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.794 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:55:22.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:22.798 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 79.285505146999995, 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:22.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 79.285465318999996, 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.804 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:55:22.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:55:22.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 79.285449303999997, 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.810 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:55:22.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:22.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 78.236320949000003, 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:22.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:55:22.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:22.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:55:22.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:55:22.822 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.822 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.822 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.822 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:22.822 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:22.822 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:22.822 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:55:22.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:22.824 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:22.822+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:22.924 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:22.922+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:22.947 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:22.947 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:22.947 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:22.947 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:22.957 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:22.963 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:22.963 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:22.994 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:22.994 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:23.022 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:23.022 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:23.025 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:23.023+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:23.043 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:23.044 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:23.044 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:23.063 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:23.064 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:23.082 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:23.082 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:23.095 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:23.095 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:23.099 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:23.099 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:23.102 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:23.103 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:23.104 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:23.126 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:23.125+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:23.227 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:23.226+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:23.256 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:23.328 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:23.327+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:23.359 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:23.359 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:23.368 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:23.429 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:23.428+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:23.471 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:23.471 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:23.479 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:23.530 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:23.527+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:23.583 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:23.591 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:23.632 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:23.630+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:23.732 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:23.730+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:23.734 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:23.734 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:23.743 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:23.833 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:23.832+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:23.887 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:23.896 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:23.934 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:23.933+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:23.966 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T22:55:23.966 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:55:23.966 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T22:55:23.966 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T22:55:23.988 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:55:24.008 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:24.009 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:24.020 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:24.061 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:24.059+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:24.136 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:24.134+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:24.139 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:24.140 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:24.140 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:24.140 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:24.142 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:24.143 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:24.143 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T22:55:24.202 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:24.202 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:24.213 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:24.237 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:24.236+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:24.278 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:24.278 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:24.278 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:24.278 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:24.278 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:24.278 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:24.278 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:24.279 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:24.279 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:24.279 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:24.279 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:24.279 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:24.281 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:24.281 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:24.282 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-14T22:55:24.320 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:24.320 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:24.329 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:24.338 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:24.337+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:24.417 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:24.417 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:24.417 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:24.417 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:24.417 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:24.417 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:24.417 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:24.418 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:24.418 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:24.418 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:24.418 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:24.418 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:24.420 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:24.421 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:24.421 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T22:55:24.439 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:24.438+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:24.477 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:24.477 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:24.486 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:24.529 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:24.532 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:24.532 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:24.533 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T22:55:24.576 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:24.574+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:24.595 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:24.595 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:24.604 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:24.647 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:24.646+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:24.663 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:24.664 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:24.664 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:24.666 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:24.666 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:24.666 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-14T22:55:24.708 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:24.708 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:24.717 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:24.750 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:24.748+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:24.837 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:24.837 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:24.838 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:24.840 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:24.841 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:24.841 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-14T22:55:24.861 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:24.861 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:24.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:24.882 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:24.880+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:24.943 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:24.940+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:24.973 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:24.973 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:24.982 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:24.997 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:55:25.024 INFO:teuthology.orchestra.run.smithi076.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:25.024 INFO:teuthology.orchestra.run.smithi076.stderr:0 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:1 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:2 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:abort 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:assert 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:bench [] [] [] [] 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs files list 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:bluefs stats 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:bluestore allocator dump block 2024-09-14T22:55:25.025 INFO:teuthology.orchestra.run.smithi076.stderr:Error EINVAL: invalid command 2024-09-14T22:55:25.028 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:25.028 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T22:55:25.044 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:25.042+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:25.091 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:25.091 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:25.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:25.103 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:55:25.148 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:25.146+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:25.225 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:25.225 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:25.241 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:25.241 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:25.244 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:25.245 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:25.245 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:25.243+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:25.253 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:25.257 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:25.257 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:25.273 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:25.273 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:25.274 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:25.290 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:25.290 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:25.306 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:25.307 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:25.323 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:25.323 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:25.326 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:25.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:25.346 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:25.345+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:25.357 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:25.357 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:25.357 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:25.357 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:25.365 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:25.447 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:25.445+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:25.508 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:25.508 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:25.508 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:25.508 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:25.508 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:25.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:25.560 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:25.559+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:25.621 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:25.621 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:25.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 153.111249996, 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:55:25.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:55:25.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:55:25.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.24659214099999, 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:55:25.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.246486205, 2024-09-14T22:55:25.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:25.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.637 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.24641150399999, 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:55:25.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:55:25.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.24632602299999, 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:55:25.642 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:55:25.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.246179637, 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:55:25.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.246129637, 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.651 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:55:25.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.655 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.245966473, 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:55:25.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:55:25.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.245928163, 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.660 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:55:25.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:25.664 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.24579449399999, 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:25.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.668 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.245713927, 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.669 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:55:25.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:55:25.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.24561699, 2024-09-14T22:55:25.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:55:25.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:55:25.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.245565435, 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:55:25.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:55:25.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.682 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.242440806, 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:55:25.686 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.242309193, 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.687 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:55:25.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:55:25.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.691 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 109.242022781, 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.692 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:55:25.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:55:25.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:25.696 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.183986527000002, 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:55:25.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:25.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:25.702 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.183946699000003, 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:25.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 82.183930684000003, 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.708 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:55:25.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:55:25.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 81.134802328999996, 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:25.714 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:25.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:55:25.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.719 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.720 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.720 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:25.720 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:25.720 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:25.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:25.722 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:25.719+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:25.822 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:25.819+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:25.827 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:25.827 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:25.827 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:25.827 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:25.836 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:25.922 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:25.920+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:25.980 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:25.980 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:25.980 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:25.980 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:25.980 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:25.988 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:26.023 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:26.022+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:26.124 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:26.123+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:26.146 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:26.146 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.294255271, 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:26.147 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:55:26.148 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.149 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.150 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.293317681, 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.151 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:55:26.154 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.29322631799999, 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.155 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:55:26.157 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:55:26.158 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.292581433, 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.159 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.160 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.161 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.162 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:26.163 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.292094054, 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:55:26.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:55:26.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.167 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.29120677900001, 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.168 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.169 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.170 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.171 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.290614355, 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.172 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.173 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.174 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.175 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.176 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.28988153900001, 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.177 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.178 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.179 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.180 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.28978239, 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:55:26.181 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:26.182 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.183 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.184 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.289463932, 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.185 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.186 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.187 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.188 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.189 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.288954438, 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.190 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.191 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:55:26.192 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.193 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:26.194 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.288896579, 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.195 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:55:26.196 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.197 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.198 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.288103956, 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.199 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:55:26.200 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.201 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.202 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.287428517, 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.203 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.204 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:55:26.205 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:55:26.206 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.287295207, 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.207 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.208 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.209 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.210 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.211 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.286678284, 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.212 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.213 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:55:26.214 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:55:26.215 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.286611427, 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.216 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.217 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:55:26.218 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.219 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.286160616, 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.220 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:55:26.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.286040383, 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:55:26.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:55:26.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 122.28472225599999, 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:26.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.233 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:26.235 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:26.235 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:26.234+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:26.335 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:26.335+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:26.338 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:26.338 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:26.338 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:26.338 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:26.346 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:26.437 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:26.436+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:26.449 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:26.458 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:26.538 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:26.537+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:26.562 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:26.562 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:26.562 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:26.562 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:26.562 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.562 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.562 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.56359446099999, 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:26.563 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:55:26.564 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.565 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.566 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.563575423, 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:55:26.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.568 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:55:26.569 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:55:26.570 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:26.571 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.563539467, 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.572 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.573 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:55:26.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:55:26.575 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.563518347, 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.576 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.577 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:55:26.578 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.579 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:55:26.580 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.562311433, 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.581 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.582 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.583 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:55:26.584 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.560481986, 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.585 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.586 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.587 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.588 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.559927699, 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.589 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.590 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.591 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.592 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:55:26.593 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.557942981, 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.594 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.595 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:55:26.596 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.597 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.598 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.55759066100001, 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.599 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.600 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.601 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.602 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.55730721800001, 2024-09-14T22:55:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.604 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:55:26.605 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:55:26.606 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.607 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.557019121, 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.608 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.609 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:26.610 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.611 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.55664429300001, 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.613 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.614 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.615 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.616 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.556323478, 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.617 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.618 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.619 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:55:26.620 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.621 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.542638243, 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:55:26.622 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.623 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.624 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:55:26.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.542303434, 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.626 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.627 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.628 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.629 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.630 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.54197327200001, 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.631 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.632 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.633 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:55:26.634 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.54175248600001, 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.635 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.637 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.638 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.639 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.541483185, 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.640 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.641 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:55:26.642 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.541296089, 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.644 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.645 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.646 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.647 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 109.54097793, 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:26.648 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:26.649 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.650 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:26.651 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:26.652 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:26.654 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:26.654 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:26.652+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:26.754 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:26.753+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:26.797 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:26.797 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:26.797 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:26.798 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:26.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:26.855 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:26.854+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:27.067 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:55:27.068 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:27.068 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:27.068 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:27.068 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:27.069 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:27.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:27.069 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:27.066+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:27.168 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:27.167+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:27.218 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 120.007413257, 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.219 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.220 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.221 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:55:27.222 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:55:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 106.883751483, 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.224 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:27.225 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.226 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:55:27.227 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.228 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 97.951984011999997, 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.229 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:55:27.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:55:27.233 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 73.042037769999993, 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:27.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.237 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 64.093377126999997, 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:55:27.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:55:27.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:55:27.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:55:27.243 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.637300250000003, 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.244 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:55:27.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:55:27.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:27.248 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.637272928000002, 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:27.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:55:27.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.636861437, 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.253 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:55:27.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.256 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.635202169999999, 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.257 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.631170746000002, 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.262 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:55:27.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:27.266 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.629327269999997, 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:55:27.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.629307595999997, 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:55:27.271 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.629268942000003, 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:55:27.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:55:27.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 60.629249062, 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:27.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:55:27.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:27.285 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.256077869000002, 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:55:27.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:27.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:55:27.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:27.291 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 56.255076574999997, 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:55:27.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:55:27.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:55:27.297 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 54.949020384000001, 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.298 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:27.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:55:27.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:55:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:211119 3.4 3:26977db1:::rbd_data.11354571ccdf.0000000000000332:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 27.332300944, 2024-09-14T22:55:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0055643710000000002, 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 211119 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883635+0000", 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.032e-06 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883640+0000", 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1649999999999997e-06 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883653+0000", 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3241000000000001e-05 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883661+0000", 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7389999999999999e-06 2024-09-14T22:55:27.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883723+0000", 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2403000000000005e-05 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883845+0000", 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012148 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:27.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.884653+0000", 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00080859200000000001 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.886346+0000", 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001692268 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889139+0000", 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027930920000000001 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889171+0000", 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2595999999999998e-05 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889198+0000", 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6763000000000001e-05 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:213874 3.3 3:c31b501c:::rbd_header.1135a4365f9c:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-14T22:55:27.310 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 10.277222686, 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0068492850000000001, 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 213874 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:27.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938714+0000", 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1059999999999998e-06 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938723+0000", 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0180000000000007e-06 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938745+0000", 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1985e-05 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938755+0000", 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0051000000000001e-05 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938843+0000", 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7671999999999997e-05 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938849+0000", 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6080000000000001e-06 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941261+0000", 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0024116459999999999 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941361+0000", 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9846999999999995e-05 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941720+0000", 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00035947399999999998 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945393+0000", 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036722679999999998 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945439+0000", 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6186000000000001e-05 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945507+0000", 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7947999999999994e-05 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945561+0000", 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4475999999999998e-05 2024-09-14T22:55:27.316 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:214566 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e148)", 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.2306940409999996, 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.98538225400000001, 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:27.317 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 214566 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:27.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985243+0000", 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.244e-06 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985254+0000", 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1428e-05 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985275+0000", 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0727e-05 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985284+0000", 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2420000000000006e-06 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985354+0000", 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9548000000000006e-05 2024-09-14T22:55:27.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985363+0000", 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1109999999999997e-06 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970284+0000", 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.98492126999999996 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.321 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970438+0000", 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00015336200000000001 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970623+0000", 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185322 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:27.322 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:27.324 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:27.323+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:27.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:27.327 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:55:27.424 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:27.423+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:27.472 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:27.472 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:27.472 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:27.472 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:27.481 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:27.487 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:27.487 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:27.504 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:27.504 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:27.521 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:27.521 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:27.525 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:27.524+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:27.537 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:27.537 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:27.537 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:27.554 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:27.554 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:27.571 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:27.571 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:27.588 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:27.588 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:27.591 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:27.592 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:27.592 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:27.592 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:27.601 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:27.642 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:27.641+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:27.727 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:27.726+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:27.744 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:27.745 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:27.753 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:27.828 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:27.827+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:27.868 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:27.868 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:27.878 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:27.930 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:27.928+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:27.982 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:27.983 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:27.991 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:28.004 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:55:28.066 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:28.065+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:28.131 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:28.130+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:28.141 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:28.141 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:28.150 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:28.232 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:28.230+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:28.253 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:28.253 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:28.262 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:28.333 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:28.332+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:28.404 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:28.404 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:28.412 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:28.456 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:28.454+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:28.515 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:28.515 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:28.523 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:28.535 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:28.533+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:28.636 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:28.634+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:28.666 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:28.666 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:28.674 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:28.758 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:28.756+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:28.818 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:28.818 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:28.827 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:28.837 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:28.836+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:28.939 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:28.937+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:28.972 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:28.972 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:28.981 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:29.040 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:29.038+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:29.115 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:29.115 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:29.124 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:29.165 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:29.164+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:29.241 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:29.240+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:29.267 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:29.267 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:29.276 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:29.360 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:29.359+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:29.419 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:29.419 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:29.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:29.442 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:29.440+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:29.543 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:29.541+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:29.572 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:29.572 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:29.580 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:29.593 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:55:29.667 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:29.666+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:29.685 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:29.686 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:29.695 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:29.744 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:29.745 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:29.745 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:29.743+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:29.762 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:29.762 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:29.778 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:29.778 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:29.794 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:29.795 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:29.795 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:29.812 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:29.812 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:29.829 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:29.829 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:29.842 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:29.842 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:29.842 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:29.842 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.84392047999999, 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.843 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:55:29.844 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:55:29.845 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.846 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.846 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:55:29.846 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:55:29.846 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.846 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.847 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.847 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:55:29.847 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:55:29.847 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.847 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.843901442, 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.848 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.849 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:55:29.850 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:55:29.851 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.852 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.843865486, 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.853 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:55:29.854 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.855 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.856 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.843844366, 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.857 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.858 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.859 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.860 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:29.861 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.84263745200001, 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:29.862 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:55:29.863 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.864 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.865 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.840808005, 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.866 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:55:29.867 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:29.868 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:55:29.869 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.840253718, 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.870 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.871 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:55:29.872 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.873 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.874 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.838269, 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.875 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.876 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.877 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.878 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.879 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.83791668000001, 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.880 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.881 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:55:29.882 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.883 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.83763323700001, 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.884 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.885 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.886 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.887 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:55:29.888 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.83734514, 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.889 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.890 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:55:29.891 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.892 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:29.893 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.83697031200001, 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.894 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:55:29.895 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:55:29.896 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.897 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.836649497, 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.898 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:55:29.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.902 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.822964262, 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:55:29.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:55:29.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.822629453, 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.907 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.908 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.909 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:55:29.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.82229929099999, 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.912 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.913 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:55:29.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:55:29.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:29.916 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.82207850499999, 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.918 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.920 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.821809204, 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.921 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:55:29.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:55:29.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.821622108, 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:55:29.925 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:55:29.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.821303949, 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:29.929 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:29.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:55:29.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:29.933 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:29.934 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:29.935 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:29.935 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:29.935 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:29.936 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:29.937 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:29.932+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:30.026 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T22:55:30.029 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T22:55:30.030 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:55:30.030 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-14T22:55:30.030 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-14T22:55:30.035 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:30.033+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:30.107 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:30.108 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:30.108 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:30.108 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:30.121 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:30.162 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:30.161+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:30.237 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:30.235+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:30.269 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:30.269 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:30.269 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:30.269 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:30.270 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:30.278 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:30.338 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:30.335+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:30.380 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-14T22:55:30.378+0000 7f721e037640 -1 mon.a@0(leader).osd e156 definitely_dead 0 2024-09-14T22:55:30.385 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:30.385 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 123.174861943, 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.386 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:55:30.387 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:55:30.388 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.389 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:55:30.390 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 110.051200169, 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.391 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:55:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:55:30.393 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 101.119432698, 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.395 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.396 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.397 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.398 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:30.399 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 76.209486455999993, 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.400 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:30.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:55:30.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:55:30.403 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:192142 3.4 3:2511efc5:::rbd_data.11358204802c.000000000000005f:head [set-alloc-hint object_size 524288 write_size 524288,write 344064~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 67.260825812999997, 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051953279999999999, 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.404 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 192142 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122558+0000", 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122559+0000", 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6670000000000001e-06 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122571+0000", 2024-09-14T22:55:30.406 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1888999999999999e-05 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122588+0000", 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7312999999999998e-05 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122597+0000", 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6370000000000005e-06 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122659+0000", 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1747000000000004e-05 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122751+0000", 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1768000000000006e-05 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:30.408 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.122835+0000", 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4190999999999994e-05 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.124641+0000", 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001806247 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127685+0000", 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0030442910000000002 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127726+0000", 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1078000000000002e-05 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:23.127753+0000", 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.65e-05 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:30.410 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.804748936000003, 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.411 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.412 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:55:30.413 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:55:30.414 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.804721614000002, 2024-09-14T22:55:30.415 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:30.416 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.417 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:55:30.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:55:30.419 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.804310123, 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:55:30.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:30.423 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.802650856, 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.424 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:30.425 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:55:30.426 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.427 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.798619432000002, 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.428 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.429 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.430 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.431 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:30.432 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.796775955999998, 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.433 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.434 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.435 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.436 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.796756281999997, 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.437 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:30.438 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.439 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.440 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:30.441 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.796717628000003, 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.442 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:55:30.444 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.445 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 63.796697748, 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.446 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.448 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:55:30.449 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:55:30.450 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.423526555000002, 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.451 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.452 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:55:30.454 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:30.455 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 59.422525260999997, 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.457 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.458 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:55:30.459 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.460 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:55:30.461 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.462 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:30.463 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 58.116469070000001, 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.464 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.465 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.466 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:55:30.467 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.468 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:211119 3.4 3:26977db1:::rbd_data.11354571ccdf.0000000000000332:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:30.469 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 30.49974963, 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0055643710000000002, 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 211119 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.470 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883635+0000", 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.032e-06 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883640+0000", 2024-09-14T22:55:30.471 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1649999999999997e-06 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883653+0000", 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3241000000000001e-05 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883661+0000", 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7389999999999999e-06 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.472 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883723+0000", 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2403000000000005e-05 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883845+0000", 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012148 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.884653+0000", 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00080859200000000001 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.473 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.886346+0000", 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001692268 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889139+0000", 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027930920000000001 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889171+0000", 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2595999999999998e-05 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.474 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889198+0000", 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6763000000000001e-05 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:213874 3.3 3:c31b501c:::rbd_header.1135a4365f9c:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:30.475 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 13.444671372, 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0068492850000000001, 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 213874 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938714+0000", 2024-09-14T22:55:30.477 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1059999999999998e-06 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938723+0000", 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0180000000000007e-06 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938745+0000", 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1985e-05 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938755+0000", 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0051000000000001e-05 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938843+0000", 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7671999999999997e-05 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938849+0000", 2024-09-14T22:55:30.479 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6080000000000001e-06 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941261+0000", 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0024116459999999999 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941361+0000", 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9846999999999995e-05 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:30.480 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941720+0000", 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00035947399999999998 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945393+0000", 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036722679999999998 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945439+0000", 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6186000000000001e-05 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.481 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945507+0000", 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7947999999999994e-05 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945561+0000", 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4475999999999998e-05 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.482 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:214566 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e148)", 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 8.3981427269999998, 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.98538225400000001, 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 214566 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.483 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985243+0000", 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.244e-06 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.484 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985254+0000", 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1428e-05 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985275+0000", 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0727e-05 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985284+0000", 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2420000000000006e-06 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985354+0000", 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9548000000000006e-05 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985363+0000", 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1109999999999997e-06 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970284+0000", 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.98492126999999996 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970438+0000", 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00015336200000000001 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970623+0000", 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185322 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.487 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:30.489 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:30.489 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:30.487+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:30.590 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:30.588+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:30.592 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:30.592 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:30.592 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:30.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:30.691 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:30.690+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:30.743 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:30.743 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:30.744 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:30.752 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:30.793 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:30.792+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:30.892 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:30.891+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99165171600001, 2024-09-14T22:55:30.898 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.899 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.900 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.901 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:55:30.902 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99163118000001, 2024-09-14T22:55:30.903 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:30.904 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.905 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:55:30.906 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.907 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:55:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:55:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.908 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99160118099999, 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.909 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.910 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.911 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.912 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99157002800001, 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.913 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.914 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:55:30.915 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:55:30.916 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.917 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99154737200001, 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.918 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.919 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:55:30.920 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.921 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.922 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99152687099999, 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.923 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:55:30.924 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.925 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.926 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99150710999999, 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.927 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.928 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:55:30.929 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:55:30.930 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.931 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99148632500001, 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.932 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:30.933 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.934 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:55:30.935 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99145391299999, 2024-09-14T22:55:30.936 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:30.937 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:55:30.938 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:55:30.939 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.940 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.991432514, 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.941 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.942 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.943 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:55:30.944 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99137111499999, 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:55:30.945 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.946 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:55:30.947 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.948 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:55:30.949 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.991331202, 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.950 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.951 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:55:30.952 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:55:30.953 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:30.954 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99130144599999, 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.955 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:55:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.957 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.958 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.991271211, 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.959 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.960 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:55:30.961 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.962 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:30.963 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.991246472, 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.964 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.965 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.966 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.967 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.991223042, 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.968 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.969 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.970 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.971 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.99112730600001, 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:55:30.972 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.973 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.974 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:55:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:55:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.975 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:55:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:55:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:55:30.976 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:55:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:55:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:55:30.977 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:55:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:55:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.978 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:55:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:55:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.979 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.991098384, 2024-09-14T22:55:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:55:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.980 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:55:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.981 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.982 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.983 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:55:30.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 128.991075059, 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.986 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.987 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:55:30.988 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.989 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:55:30.990 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 105.858377316, 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:30.991 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:30.992 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:55:30.993 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:55:30.994 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.995 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:30.996 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:30.996 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:30.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:30.997 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:30.995+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is set 2024-09-14T22:55:31.032 INFO:teuthology.orchestra.run.smithi076.stderr:marked in osd.1. 2024-09-14T22:55:31.049 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-14T22:55:31.097 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:31.096+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:31.117 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:31.117 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:31.117 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:31.126 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:31.198 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:31.197+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:31.228 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:31.228 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:31.228 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:31.229 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:31.229 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:31.237 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:31.299 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:31.297+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:31.341 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 119.443536497, 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:31.342 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:31.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:55:31.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:55:31.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:55:31.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.81774237800001, 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.349 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:55:31.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.817654386, 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:55:31.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:55:31.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.817420469, 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.358 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:55:31.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:55:31.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.362 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.817239574, 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.363 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:55:31.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:55:31.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.816940071, 2024-09-14T22:55:31.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:55:31.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.371 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.816593861, 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:55:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:55:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:55:31.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.378 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.816230133, 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:55:31.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:55:31.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.815640516, 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.383 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.81543827900001, 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.388 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:55:31.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.81524846799999, 2024-09-14T22:55:31.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:31.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:55:31.396 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.814936465, 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:55:31.397 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:55:31.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.401 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.81469397799999, 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.402 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:55:31.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.406 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.81453501599999, 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:55:31.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:55:31.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.411 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.813900746, 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:55:31.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:55:31.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.813686636, 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.416 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:31.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:55:31.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.813550166, 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.421 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:31.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:31.425 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 113.813297686, 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:55:31.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:55:31.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.714399362000002, 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:31.430 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:55:31.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:55:31.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.436 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.218912733000003, 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:31.437 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:31.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:55:31.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:55:31.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.443 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.443 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.443 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.443 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:31.444 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:31.445 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:31.443+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:31.545 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:31.544+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:31.588 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:31.589 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:31.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:31.646 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:31.645+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:31.699 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:31.700 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:31.709 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:31.753 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:31.752+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.73006508399999, 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.814 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72984718800001, 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:55:31.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.727163542, 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.824 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:55:31.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:55:31.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72700512700001, 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.828 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:55:31.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72697786099999, 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.833 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:55:31.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72677071999999, 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:55:31.837 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:55:31.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.726736456, 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.842 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:55:31.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72510441, 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:55:31.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:55:31.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.725079798, 2024-09-14T22:55:31.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:55:31.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:55:31.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72494551200001, 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.856 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:55:31.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.860 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72379361899999, 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.861 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:55:31.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.723769715, 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.866 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:55:31.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72373619499999, 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.870 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:55:31.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72358154899999, 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:55:31.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:55:31.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.879 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.723540375, 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:55:31.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:55:31.883 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.723512215, 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.884 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 129.72332282400001, 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:55:31.888 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:31.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 127.807232997, 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.893 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:31.897 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 127.807213166, 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.898 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:55:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:55:31.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:55:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:55:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:55:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:55:31.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.607506841000003, 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:55:31.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:55:31.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:31.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:55:31.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:55:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:55:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:31.912 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:31.912 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:31.914 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:31.915 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:31.912+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:31.936 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T22:55:32.014 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:32.013+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:32.062 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:32.062 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:32.062 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:32.071 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:32.079 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:32.080 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:32.097 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:32.097 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:32.113 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:32.113 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:32.115 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:32.113+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:32.129 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:32.129 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:32.129 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:32.146 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:32.162 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:32.162 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:32.179 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:32.179 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:32.182 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:32.182 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:32.215 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:32.215 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:32.224 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:32.225 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:32.223+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:32.317 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:32.316+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:32.328 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:32.328 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:32.336 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:32.418 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:32.417+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:32.439 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:32.440 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:32.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:32.519 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:32.517+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:32.551 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:32.551 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:32.559 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:32.643 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:32.642+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:32.703 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:32.704 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:32.712 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:32.721 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:32.719+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:32.821 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:32.820+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:32.856 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:32.856 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:32.865 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:32.924 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:32.921+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:32.969 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:32.970 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:32.978 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:33.025 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:33.023+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:33.050 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T22:55:33.126 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:33.124+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:33.153 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:33.153 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:33.165 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:33.227 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:33.226+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:33.310 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:33.310 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:33.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:33.328 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:33.327+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:33.425 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:33.425 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:33.428 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:33.427+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:33.434 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:33.530 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:33.528+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:33.578 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:33.578 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:33.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:33.630 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:33.629+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:33.688 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:33.689 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:33.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:33.731 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:33.729+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:33.802 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:33.802 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:33.810 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:33.854 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:33.853+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:33.913 INFO:teuthology.orchestra.run.smithi076.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:33.913 INFO:teuthology.orchestra.run.smithi076.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:33.922 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:33.962 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:33.961+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:34.033 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:34.032+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:34.036 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:34.036 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:34.047 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:34.048 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is set 2024-09-14T22:55:34.134 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:34.133+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:34.156 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:34.156 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:34.165 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:34.184 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T22:55:34.235 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:34.234+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:34.308 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:34.308 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:34.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:34.361 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:34.360+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:34.420 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:34.420 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:34.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:34.437 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-14T22:55:34.436+0000 7f236a713640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33340) UID: 0 2024-09-14T22:55:34.504 INFO:teuthology.orchestra.run.smithi076.stdout:osd.0: {} 2024-09-14T22:55:34.505 INFO:teuthology.orchestra.run.smithi076.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T22:55:34.521 INFO:teuthology.orchestra.run.smithi076.stdout:osd.1: {} 2024-09-14T22:55:34.521 INFO:teuthology.orchestra.run.smithi076.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T22:55:34.537 INFO:teuthology.orchestra.run.smithi076.stdout:osd.2: {} 2024-09-14T22:55:34.537 INFO:teuthology.orchestra.run.smithi076.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T22:55:34.538 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:34.536+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:34.553 INFO:teuthology.orchestra.run.smithi076.stdout:osd.3: {} 2024-09-14T22:55:34.553 INFO:teuthology.orchestra.run.smithi076.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T22:55:34.553 INFO:teuthology.orchestra.run.smithi076.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-14T22:55:34.569 INFO:teuthology.orchestra.run.smithi076.stdout:osd.5: {} 2024-09-14T22:55:34.570 INFO:teuthology.orchestra.run.smithi076.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T22:55:34.573 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-14T22:55:34.573 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T22:55:34.582 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:34.586 INFO:teuthology.orchestra.run.smithi076.stdout:osd.6: {} 2024-09-14T22:55:34.586 INFO:teuthology.orchestra.run.smithi076.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T22:55:34.602 INFO:teuthology.orchestra.run.smithi076.stdout:osd.7: {} 2024-09-14T22:55:34.602 INFO:teuthology.orchestra.run.smithi076.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T22:55:34.606 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T22:55:34.606 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T22:55:34.666 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:34.665+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.64301522299999, 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.727 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.731 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.64279732700001, 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.640113681, 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.736 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:55:34.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:55:34.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.739 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.63995526599999, 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.740 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:55:34.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.639928, 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:55:34.744 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.748 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.63972085899999, 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.749 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:55:34.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:55:34.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.639686595, 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.638054549, 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:55:34.757 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:55:34.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.638029937, 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.762 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:55:34.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:34.766 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.63789565100001, 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:34.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:55:34.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:55:34.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.63674375799999, 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.771 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:34.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.775 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.63671985400001, 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:55:34.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:55:34.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.63668633399999, 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:55:34.780 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:55:34.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.63653168799999, 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.785 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.636490514, 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:55:34.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.636462354, 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:55:34.794 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.798 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 132.63627296300001, 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.799 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:55:34.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 130.720183136, 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:55:34.803 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.807 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 130.720163305, 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.808 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:55:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 54.520456979999999, 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:55:34.812 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:55:34.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:55:34.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:34.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:34.818 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:34.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:34.820 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:34.818+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:34.920 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:34.919+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:34.962 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:34.962 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:34.963 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:34.963 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:34.971 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:35.020 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:35.019+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:35.080 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:35.080 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:35.080 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:35.081 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:35.081 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:35.090 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:35.131 INFO:tasks.ceph.osd.5.smithi114.stderr:2024-09-14T22:55:35.130+0000 7faddba4f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33808) UID: 0 2024-09-14T22:55:35.223 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:35.222+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:35.234 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:35.234 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:35.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:35.234 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:35.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.234 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 162.72439945299999, 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:55:35.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:55:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:35.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.241 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.859741598, 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:55:35.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.859635662, 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.246 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:35.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.250 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.859560961, 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:55:35.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.85947548, 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:55:35.255 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:55:35.259 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.859329094, 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.260 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:55:35.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.859279094, 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.265 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:55:35.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:55:35.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.85911593, 2024-09-14T22:55:35.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.273 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.85907761999999, 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.274 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:35.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:55:35.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.278 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.858943951, 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:55:35.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:55:35.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.858863384, 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.284 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:35.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.85876644699999, 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:55:35.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.85871489199999, 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.293 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.855590263, 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:55:35.297 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.301 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.85545865, 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:55:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:55:35.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 118.85517223799999, 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.306 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:55:35.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.797135983999993, 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:55:35.310 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:55:35.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:55:35.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.797096155999995, 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:55:35.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:55:35.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.321 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 91.797080140999995, 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.322 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:55:35.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:35.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 90.747951786000002, 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:55:35.328 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:35.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:55:35.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.334 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:35.335 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:35.335 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:35.336 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T22:55:35.337 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:35.334+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:35.437 INFO:tasks.ceph.osd.7.smithi114.stderr:2024-09-14T22:55:35.435+0000 7f2f42bc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33814) UID: 0 2024-09-14T22:55:35.440 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:35.440 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:35.440 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:35.440 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:35.449 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T22:55:35.537 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:35.536+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:35.551 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:35.551 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:35.551 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:35.551 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:35.551 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:35.560 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T22:55:35.644 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-14T22:55:35.643+0000 7f747af54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33330) UID: 0 2024-09-14T22:55:35.705 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:35.705 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58783 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 251904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.853159557, 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104046869, 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58783 2024-09-14T22:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.707 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849725+0000", 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999998e-07 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849728+0000", 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-06 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849734+0000", 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8709999999999999e-06 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.708 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.849748+0000", 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3534e-05 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953511+0000", 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103762934 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953587+0000", 2024-09-14T22:55:35.709 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6131999999999995e-05 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.953772+0000", 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00018457099999999999 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58797 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 425984~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.85222196699999, 2024-09-14T22:55:35.710 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104468752, 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58797 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.711 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850662+0000", 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850663+0000", 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.712 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850668+0000", 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0030000000000002e-06 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850674+0000", 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6159999999999998e-06 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850679+0000", 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8300000000000003e-06 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.713 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954628+0000", 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103949056 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.954755+0000", 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012702899999999999 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955131+0000", 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037577800000000001 2024-09-14T22:55:35.714 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58807 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 75776~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.852130604, 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104788306, 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.715 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58807 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:35.716 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850754+0000", 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3399999999999999e-07 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850757+0000", 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.413e-06 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.717 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850763+0000", 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4810000000000003e-06 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.850768+0000", 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8269999999999997e-06 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955005+0000", 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104237496 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955074+0000", 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8418999999999997e-05 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955542+0000", 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046813600000000002 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58850 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 163840~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:35.719 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.85148571900001, 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10474724000000001, 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58850 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851399+0000", 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851774+0000", 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000375919 2024-09-14T22:55:35.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851794+0000", 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.914e-05 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851803+0000", 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1710000000000008e-06 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851808+0000", 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7389999999999999e-06 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955571+0000", 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10376371400000001 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.955640+0000", 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8757999999999995e-05 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956146+0000", 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00050579900000000003 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58863 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 514048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.85099833999999, 2024-09-14T22:55:35.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10464599500000001, 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58863 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851886+0000", 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6899999999999999e-07 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851888+0000", 2024-09-14T22:55:35.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.567e-06 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851891+0000", 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4450000000000001e-06 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.851902+0000", 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0370999999999999e-05 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956177+0000", 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10427507799999999 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956247+0000", 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0554000000000002e-05 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956532+0000", 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028471099999999999 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.728 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58872 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 337920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.85011106499999, 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10410148700000001, 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.729 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58872 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852773+0000", 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852774+0000", 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2699999999999999e-07 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852783+0000", 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6430000000000007e-06 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852791+0000", 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0469999999999994e-06 2024-09-14T22:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.852797+0000", 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1699999999999996e-06 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956561+0000", 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103764 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956630+0000", 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8967000000000006e-05 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.956875+0000", 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024513300000000003 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58916 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:35.733 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.849518641, 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104209072, 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58916 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853366+0000", 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8699999999999995e-07 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853368+0000", 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2539999999999999e-06 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853375+0000", 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5339999999999996e-06 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.853381+0000", 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8660000000000002e-06 2024-09-14T22:55:35.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957250+0000", 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103869502 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957317+0000", 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6917999999999999e-05 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957575+0000", 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025741099999999998 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58920 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.84878582499999, 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103944444, 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.738 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58920 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854098+0000", 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854099+0000", 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7100000000000002e-07 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854102+0000", 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6560000000000001e-06 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854108+0000", 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.781e-06 2024-09-14T22:55:35.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854114+0000", 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1109999999999999e-06 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957608+0000", 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10349480599999999 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.957678+0000", 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9227000000000007e-05 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958043+0000", 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00036529200000000003 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.742 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58941 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 81920~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.848686676, 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10443919, 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58941 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854198+0000", 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5200000000000002e-07 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854200+0000", 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9439999999999999e-06 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854206+0000", 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7939999999999999e-06 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854219+0000", 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3447e-05 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.745 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958073+0000", 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103853731 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958144+0000", 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0578000000000001e-05 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958637+0000", 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000493244 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.746 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58947 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 432128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.84836821799999, 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104682386, 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58947 2024-09-14T22:55:35.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854516+0000", 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854517+0000", 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9299999999999998e-07 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854521+0000", 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.882e-06 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854527+0000", 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8279999999999999e-06 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.854532+0000", 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3370000000000001e-06 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958667+0000", 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104135802 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.958736+0000", 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8915e-05 2024-09-14T22:55:35.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959198+0000", 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00046212899999999998 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58966 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.84785872399999, 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104951526, 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.751 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58966 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855026+0000", 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5600000000000001e-07 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855028+0000", 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3080000000000002e-06 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855034+0000", 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0990000000000004e-06 2024-09-14T22:55:35.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855039+0000", 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7899999999999999e-06 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959229+0000", 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104189426 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959299+0000", 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0765000000000004e-05 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.959977+0000", 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00067768199999999998 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58976 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 124928~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.84780086500001, 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105473634, 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58976 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855083+0000", 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855161+0000", 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8048999999999998e-05 2024-09-14T22:55:35.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855164+0000", 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.9409999999999999e-06 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855173+0000", 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4319999999999999e-06 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855179+0000", 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9689999999999999e-06 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960140+0000", 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104961479 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960354+0000", 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000213539 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960557+0000", 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020322499999999999 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:58990 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 475136~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.84700824199999, 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.105014239, 2024-09-14T22:55:35.761 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 58990 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:35.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.855876+0000", 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856460+0000", 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00058358300000000004 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856464+0000", 2024-09-14T22:55:35.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0579999999999999e-06 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856476+0000", 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2048000000000001e-05 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856492+0000", 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6742999999999999e-05 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960589+0000", 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104096205 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960660+0000", 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1105000000000004e-05 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960890+0000", 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000230497 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.765 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59001 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 301056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.846332803, 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10461239899999999, 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59001 2024-09-14T22:55:35.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856551+0000", 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.767 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856552+0000", 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2e-07 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856562+0000", 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6749999999999997e-06 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856568+0000", 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7009999999999999e-06 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856573+0000", 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8740000000000003e-06 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960920+0000", 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10434659 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.960990+0000", 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9979000000000002e-05 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961164+0000", 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00017406 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59020 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 169984~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.846199493, 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10481296, 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.770 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59020 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856685+0000", 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8700000000000001e-07 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856687+0000", 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.007e-06 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856694+0000", 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3459999999999999e-06 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.856698+0000", 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7720000000000002e-06 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961193+0000", 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10449425800000001 2024-09-14T22:55:35.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961265+0000", 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2675999999999993e-05 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961498+0000", 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023251399999999999 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.774 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59027 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 518144~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.84558257, 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104462285, 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59027 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857302+0000", 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3600000000000003e-07 2024-09-14T22:55:35.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857305+0000", 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8870000000000001e-06 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857313+0000", 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8879999999999997e-06 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857319+0000", 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.5600000000000001e-06 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961527+0000", 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104208764 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961597+0000", 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9641000000000006e-05 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961764+0000", 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000166909 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59038 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 344064~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.845515713, 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10470481600000001, 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.779 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59038 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857369+0000", 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5400000000000002e-07 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857371+0000", 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857378+0000", 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3929999999999996e-06 2024-09-14T22:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857383+0000", 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0259999999999998e-06 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961799+0000", 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104415967 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.961869+0000", 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0397999999999999e-05 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962073+0000", 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000204358 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59054 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 212992~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.84506490199999, 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10458139800000001, 2024-09-14T22:55:35.783 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59054 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857819+0000", 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857820+0000", 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2900000000000003e-07 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857823+0000", 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6570000000000001e-06 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857840+0000", 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6861e-05 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857845+0000", 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8599999999999998e-06 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962105+0000", 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104259148 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962175+0000", 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0135999999999994e-05 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962401+0000", 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000226107 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59073 3.1b 3:df71c2af:::rbd_data.113535cec067.0000000000000029:8 [read 38912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.787 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.844944669, 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.104758105, 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59073 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.857940+0000", 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858954+0000", 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001014205 2024-09-14T22:55:35.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858958+0000", 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5009999999999998e-06 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858970+0000", 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1535000000000001e-05 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.858977+0000", 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3350000000000001e-06 2024-09-14T22:55:35.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962432+0000", 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103454897 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962508+0000", 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5684000000000002e-05 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.962698+0000", 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000189948 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:59100 3.1b 3:da23d89d:::rbd_data.113535cec067.0000000000000006:8 [read 256000~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 131.84362654200001, 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.10414359500000001, 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:35.792 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 59100 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:35.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859258+0000", 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3000000000000001e-07 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859260+0000", 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2199999999999999e-06 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:35.794 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859266+0000", 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0610000000000001e-06 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.859271+0000", 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8080000000000004e-06 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963068+0000", 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.103796897 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:35.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963135+0000", 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7175000000000007e-05 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:23.963401+0000", 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026600400000000002 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:35.796 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:35.798 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T22:55:35.798 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:35.796+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:35.898 INFO:tasks.ceph.osd.6.smithi114.stderr:2024-09-14T22:55:35.897+0000 7f38855ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33811) UID: 0 2024-09-14T22:55:35.942 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:35.942 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:35.942 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:35.942 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:35.951 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T22:55:35.998 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-14T22:55:35.997+0000 7f251d15f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33344) UID: 0 2024-09-14T22:55:36.050 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [4] live_osds: [0, 1, 2, 3, 5, 6, 7] 2024-09-14T22:55:36.050 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T22:55:36.050 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T22:55:36.050 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T22:55:36.055 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:36.055 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:36.055 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:36.066 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:55:36.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T22:55:36.110 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-14T22:55:36.109+0000 7fcd4d3ce640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33324) UID: 0 2024-09-14T22:55:36.152 INFO:teuthology.orchestra.run.smithi076.stdout:All operations completed A-OK! 2024-09-14T22:55:36.168 DEBUG:teuthology.parallel:result is None 2024-09-14T22:55:36.168 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T22:55:36.177 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T22:55:36.214 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T22:55:36.228 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122383 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3510272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.229297448, 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079848851999999998, 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.229 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122383 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996543+0000", 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.230 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996544+0000", 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6600000000000002e-07 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996546+0000", 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.108e-06 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.231 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996552+0000", 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0630000000000002e-06 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996557+0000", 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6820000000000002e-06 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076148+0000", 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079591767999999993 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.232 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076225+0000", 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6481000000000002e-05 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076392+0000", 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000167384 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122384 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:36.233 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.22927841000001, 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080113413999999994, 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122384 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:36.234 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996562+0000", 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996563+0000", 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1800000000000001e-07 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.235 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996565+0000", 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0999999999999998e-06 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996571+0000", 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1739999999999997e-06 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996576+0000", 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8049999999999997e-06 2024-09-14T22:55:36.236 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076423+0000", 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079846808000000005 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076478+0000", 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4982999999999999e-05 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076676+0000", 2024-09-14T22:55:36.237 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00019812599999999999 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122385 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3518464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.229242454, 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080444320999999999, 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.238 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122385 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996598+0000", 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.239 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996599+0000", 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.82e-07 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996601+0000", 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.4420000000000001e-06 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996608+0000", 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7789999999999998e-06 2024-09-14T22:55:36.240 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996613+0000", 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0010000000000001e-06 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076723+0000", 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080109763 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.076835+0000", 2024-09-14T22:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000111748 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077043+0000", 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000208106 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122386 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4169728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:36.242 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.229221334, 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080729906000000004, 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122386 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:36.243 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.996619+0000", 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997785+0000", 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0011660209999999999 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.244 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997791+0000", 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8760000000000002e-06 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997804+0000", 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2228999999999999e-05 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997812+0000", 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1820000000000006e-06 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.245 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077082+0000", 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079270698000000001 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077164+0000", 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1771e-05 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077349+0000", 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185129 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122387 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.22801441999999, 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905375000000001, 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122387 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.247 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.997826+0000", 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998393+0000", 2024-09-14T22:55:36.248 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00056654000000000003 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998398+0000", 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2390000000000002e-06 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998406+0000", 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2099999999999993e-06 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.249 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.998412+0000", 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3900000000000001e-06 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077392+0000", 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.078979826000000003 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077482+0000", 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0117000000000005e-05 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.250 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.077732+0000", 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025005300000000003 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122394 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3559424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.226184973, 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079877620999999996, 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.251 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122394 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999656+0000", 2024-09-14T22:55:36.252 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999936+0000", 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028034300000000001 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999943+0000", 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2729999999999997e-06 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.253 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999952+0000", 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2309999999999996e-06 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:36.999958+0000", 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079211+0000", 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079253174999999995 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079300+0000", 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.8599999999999999e-05 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079533+0000", 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023389 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.255 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122398 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3575808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.225630686, 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079892582000000004, 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.256 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122398 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000210+0000", 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000490+0000", 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00028039100000000001 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000494+0000", 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.4170000000000001e-06 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000502+0000", 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1149999999999994e-06 2024-09-14T22:55:36.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.000512+0000", 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0366999999999999e-05 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079812+0000", 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079299517999999999 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.079884+0000", 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2430999999999998e-05 2024-09-14T22:55:36.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.080103+0000", 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021834299999999999 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122414 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3633152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.223645968, 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079931071000000006, 2024-09-14T22:55:36.260 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122414 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.261 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002195+0000", 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002514+0000", 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00031938099999999998 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002519+0000", 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5730000000000001e-06 2024-09-14T22:55:36.262 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002528+0000", 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1679999999999993e-06 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002534+0000", 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4540000000000004e-06 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081831+0000", 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079297119999999999 2024-09-14T22:55:36.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.081890+0000", 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8788e-05 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082126+0000", 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023558699999999999 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122416 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3641344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.22329364799999, 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079926611999999994, 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122416 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.265 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002547+0000", 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002804+0000", 2024-09-14T22:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00025667399999999998 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002808+0000", 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5299999999999998e-06 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002817+0000", 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2230000000000007e-06 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002821+0000", 2024-09-14T22:55:36.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6380000000000003e-06 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082166+0000", 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079344578999999998 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082258+0000", 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2163000000000005e-05 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082474+0000", 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021580500000000001 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122418 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3649536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.22301020499999, 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079921690000000004, 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.269 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122418 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.002831+0000", 2024-09-14T22:55:36.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003093+0000", 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026204700000000001 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003096+0000", 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8770000000000003e-06 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003103+0000", 2024-09-14T22:55:36.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3099999999999997e-06 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003107+0000", 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.1729999999999997e-06 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082522+0000", 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079415071000000004 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082574+0000", 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1984000000000002e-05 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082752+0000", 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000178228 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122420 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3657728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:36.273 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.222722108, 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079897996999999998, 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122420 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003119+0000", 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003459+0000", 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00034063399999999998 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003467+0000", 2024-09-14T22:55:36.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7500000000000003e-06 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003476+0000", 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4050000000000006e-06 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003483+0000", 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.276 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082798+0000", 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079315595000000003 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.082849+0000", 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0998999999999997e-05 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083017+0000", 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016750399999999999 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.277 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122422 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3665920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.22234727999999, 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079873509999999995, 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.278 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122422 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003493+0000", 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003761+0000", 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026721699999999999 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003765+0000", 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0960000000000003e-06 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003793+0000", 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8402999999999999e-05 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003801+0000", 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9950000000000005e-06 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083073+0000", 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079272100999999998 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083151+0000", 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7992999999999997e-05 2024-09-14T22:55:36.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083367+0000", 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021570500000000001 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122469 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 3936256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.222026465, 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079905799, 2024-09-14T22:55:36.282 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122469 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.003814+0000", 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004100+0000", 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000285462 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004105+0000", 2024-09-14T22:55:36.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.7870000000000001e-06 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004115+0000", 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0013999999999999e-05 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.004122+0000", 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9949999999999999e-06 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.285 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083430+0000", 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079308045999999993 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083511+0000", 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1524000000000001e-05 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.083720+0000", 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00020897099999999999 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.286 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122517 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4915200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.20834123, 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079993964000000001, 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.287 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122517 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017500+0000", 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017797+0000", 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029783000000000001 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017806+0000", 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0839999999999998e-06 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017814+0000", 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.6860000000000006e-06 2024-09-14T22:55:36.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017821+0000", 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097132+0000", 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079310694000000001 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097226+0000", 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3702999999999994e-05 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097493+0000", 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00026798300000000001 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122518 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4923392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:36.291 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.20800642099999, 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080051793999999996, 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122518 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:36.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.017834+0000", 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018130+0000", 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000295674 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018134+0000", 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4530000000000004e-06 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018144+0000", 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3249999999999997e-06 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018153+0000", 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2129999999999997e-06 2024-09-14T22:55:36.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097583+0000", 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079429851999999995 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097675+0000", 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2597999999999997e-05 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097886+0000", 2024-09-14T22:55:36.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021067899999999999 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122519 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4116480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.207676259, 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080087625999999995, 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.296 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122519 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018165+0000", 2024-09-14T22:55:36.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018357+0000", 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000192153 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018361+0000", 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0339999999999998e-06 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018370+0000", 2024-09-14T22:55:36.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3090000000000001e-06 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018376+0000", 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2260000000000002e-06 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.097933+0000", 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079557244999999999 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098031+0000", 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7429999999999994e-05 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098252+0000", 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00022122900000000001 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.300 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122520 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4931584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.207455473, 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080258711999999996, 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122520 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018385+0000", 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018624+0000", 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023909100000000001 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018629+0000", 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5369999999999999e-06 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018641+0000", 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2071e-05 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018646+0000", 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0220000000000004e-06 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098312+0000", 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079666415000000004 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098402+0000", 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9751000000000002e-05 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098644+0000", 2024-09-14T22:55:36.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00024182500000000001 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122521 3.4 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:c [read 4124672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.20718617199999, 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080329932000000007, 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.305 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122521 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018655+0000", 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018817+0000", 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000162466 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018821+0000", 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9129999999999996e-06 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018828+0000", 2024-09-14T22:55:36.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7089999999999996e-06 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018832+0000", 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9650000000000002e-06 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098687+0000", 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079855182999999996 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098774+0000", 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7136000000000001e-05 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.098985+0000", 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021055999999999999 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122522 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:36.309 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.206999076, 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.080188380000000004, 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122522 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.018842+0000", 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019129+0000", 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000286901 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019132+0000", 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.8720000000000004e-06 2024-09-14T22:55:36.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019145+0000", 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2648e-05 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019150+0000", 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.101e-06 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099019+0000", 2024-09-14T22:55:36.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079868653999999997 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099030+0000", 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1204e-05 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:122523 3.4 3.24620344 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:36.313 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 119.206680917, 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079902360000000006, 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 122523 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019160+0000", 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019306+0000", 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014642400000000001 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019310+0000", 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6210000000000001e-06 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019317+0000", 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8800000000000002e-06 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.019324+0000", 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7200000000000006e-06 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099052+0000", 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.079728006000000004 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:37.099062+0000", 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7089999999999997e-06 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.317 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:36.318 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout: "epoch": 162, 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout: "fsid": "9a42a40e-8d25-4801-9835-f823a294bdbe", 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout: "created": "2024-09-14T22:52:37.944451+0000", 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout: "modified": "2024-09-14T22:55:36.050376+0000", 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout: "last_up_change": "2024-09-14T22:54:57.788789+0000", 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout: "last_in_change": "2024-09-14T22:55:31.022383+0000", 2024-09-14T22:55:36.368 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_num": 5806080, 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_set": [ 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "nodeep-scrub", 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "noscrub", 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "pglog_hardlimit", 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "purged_snapdirs", 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_deletes", 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "sortbitwise" 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_version": 5, 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T22:55:36.369 INFO:teuthology.orchestra.run.smithi076.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_snapshot": "", 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_max": 3, 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "max_osd": 8, 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "require_min_compat_client": "luminous", 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "min_compat_client": "luminous", 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "require_osd_release": "squid", 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "allow_crimson": false, 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "pools": [ 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 1, 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": ".mgr", 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:50.813186+0000", 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 1, 2024-09-14T22:55:36.370 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool", 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 1, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 1, 2024-09-14T22:55:36.371 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 1, 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 1, 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 1, 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "18", 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:55:36.372 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 0, 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 0, 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:55:36.373 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:55:36.374 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "options": { 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_max": 32, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_min": 1 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.375 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": {} 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 8, 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 8, 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.25, 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2, 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2, 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.376 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 2, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "rbd", 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:52:55.912547+0000", 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:55:36.377 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 8, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 8, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 8, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 8, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 8, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:55:36.378 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "22", 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 2, 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 22, 2024-09-14T22:55:36.379 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:55:36.380 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:55:36.381 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 2.9900000095367432, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 2.9900000095367432, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:55:36.382 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_name": "pool_client.0", 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "create_time": "2024-09-14T22:53:05.643169+0000", 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "flags": 8193, 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "type": 1, 2024-09-14T22:55:36.383 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 2, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "min_size": 1, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_rule": 0, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_count": 0, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_target": 0, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "is_stretch_pool": false, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "object_hash": 2, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_autoscale_mode": "off", 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num": 32, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num": 32, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_placement_num_target": 32, 2024-09-14T22:55:36.384 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_target": 32, 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_num_pending": 32, 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "last_pg_merge_meta": { 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "source_pgid": "0.0", 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "ready_epoch": 0, 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_started": 0, 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "last_epoch_clean": 0, 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "source_version": "0'0", 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "target_version": "0'0" 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "last_change": "162", 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend": "0", 2024-09-14T22:55:36.385 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "auid": 0, 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_mode": "selfmanaged", 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_seq": 47, 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "snap_epoch": 162, 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "pool_snaps": [], 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps": "[]", 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_bytes": 0, 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "quota_max_objects": 0, 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "tiers": [], 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "tier_of": -1, 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "read_tier": -1, 2024-09-14T22:55:36.386 INFO:teuthology.orchestra.run.smithi076.stdout: "write_tier": -1, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_mode": "none", 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_bytes": 0, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "target_max_objects": 0, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_flush_age": 0, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "cache_min_evict_age": 0, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profile": "", 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_params": { 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "none" 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.387 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_period": 0, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_count": 0, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "use_gmt_hitset": true, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "min_read_recency_for_promote": 0, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "min_write_recency_for_promote": 0, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "hit_set_search_last_n": 0, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "grade_table": [], 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "stripe_width": 0, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "expected_num_objects": 0, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "fast_read": false, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "options": {}, 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "application_metadata": { 2024-09-14T22:55:36.388 INFO:teuthology.orchestra.run.smithi076.stdout: "rbd": {} 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "read_balance": { 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "score_type": "Fair distribution", 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "score_acting": 2.2400000095367432, 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "score_stable": 2.2400000095367432, 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity_weighted": 0.83006668090820312, 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity": 0.82999998331069946, 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.389 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.390 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:55:36.390 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.390 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:55:36.390 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "45f61e20-37e6-4a13-b005-1f135f5e5a58", 2024-09-14T22:55:36.390 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:55:36.390 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:55:36.391 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.5721282958984375, 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 157, 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6800", 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6801", 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6802", 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6803", 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:55:36.394 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6806", 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6807", 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.395 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6804", 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6805", 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2488220599 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6801/2488220599", 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6803/2488220599", 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6807/2488220599", 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6805/2488220599", 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:55:36.397 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "bc72773d-098f-4ac5-89b5-13dae87b65b4", 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 0.0684051513671875, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 158, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.398 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6808", 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6809", 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6810", 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6811", 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6814", 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6815", 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:55:36.401 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6812", 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6813", 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 522829438 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6809/522829438", 2024-09-14T22:55:36.402 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6811/522829438", 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6815/522829438", 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6813/522829438", 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "2c2a50b0-e6ab-4593-9a26-badf1cc3dd34", 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:55:36.403 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 160, 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6824", 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6825", 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6826", 2024-09-14T22:55:36.405 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6827", 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.406 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6830", 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6831", 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6828", 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6829", 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3214755929 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6825/3214755929", 2024-09-14T22:55:36.408 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6827/3214755929", 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6831/3214755929", 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6829/3214755929", 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "40d20d5f-5509-4a9f-8b13-c9e8da7339bc", 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:55:36.409 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 157, 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6816", 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.410 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6817", 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6818", 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.411 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6819", 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6822", 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.412 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6823", 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6820", 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.413 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.76:6821", 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2699236014 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.76:6817/2699236014", 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.76:6819/2699236014", 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.76:6823/2699236014", 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.76:6821/2699236014", 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:55:36.414 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "78bf5206-fbf7-4668-a27f-9eb867edc4e5", 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 0, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 86, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 124, 2024-09-14T22:55:36.415 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6824", 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6825", 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6826", 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6827", 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.417 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6830", 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6831", 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.418 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6828", 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6829", 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2097675638 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.419 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6825/2097675638", 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6827/2097675638", 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6831/2097675638", 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6829/2097675638", 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "exists" 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "72838af1-36f2-4cf6-a5ee-6daf5d4ea70a", 2024-09-14T22:55:36.420 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 157, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.421 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6800", 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6801", 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.422 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6802", 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6803", 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.423 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6806", 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6807", 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:55:36.424 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6804", 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6805", 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3469663218 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6801/3469663218", 2024-09-14T22:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6803/3469663218", 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6807/3469663218", 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6805/3469663218", 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "11be725a-e81f-4b8a-8a68-acb54e67b443", 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:55:36.426 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 124, 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.427 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6816", 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6817", 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:55:36.428 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6818", 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6819", 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:55:36.429 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6822", 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6823", 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.430 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6820", 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6821", 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2551471767 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.431 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6817/2551471767", 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6819/2551471767", 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6823/2551471767", 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6821/2551471767", 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:55:36.432 INFO:teuthology.orchestra.run.smithi076.stdout: "uuid": "94103011-716e-495f-aeb3-759071ce6f14", 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "up": 1, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "in": 1, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "weight": 1, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_affinity": 1, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_begin": 0, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "last_clean_end": 0, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "up_from": 14, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "up_thru": 157, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "down_at": 0, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "lost_at": 0, 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addrs": { 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.433 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6808", 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6809", 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addrs": { 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6810", 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6811", 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addrs": { 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.435 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6814", 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6815", 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addrs": { 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: "addrvec": [ 2024-09-14T22:55:36.436 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v2", 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6812", 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "172.21.15.114:6813", 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 941014947 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.437 INFO:teuthology.orchestra.run.smithi076.stdout: "public_addr": "172.21.15.114:6809/941014947", 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_addr": "172.21.15.114:6811/941014947", 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_back_addr": "172.21.15.114:6815/941014947", 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "heartbeat_front_addr": "172.21.15.114:6813/941014947", 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "state": [ 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "exists", 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "up" 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_xinfo": [ 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 0, 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:55:36.438 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.133153+0000", 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 1, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.415439+0000", 2024-09-14T22:55:36.439 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 2, 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.880448+0000", 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 3, 2024-09-14T22:55:36.440 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.879405+0000", 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 4, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:55:36.441 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:47.079602+0000", 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 124 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 5, 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.493580+0000", 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.442 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 6, 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.693553+0000", 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": 7, 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "down_stamp": "0.000000", 2024-09-14T22:55:36.443 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_probability": 0, 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "laggy_interval": 0, 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "features": 4540701547738038271, 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "old_weight": 0, 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "last_purged_snaps_scrub": "2024-09-14T22:52:46.442044+0000", 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "dead_epoch": 0 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap": [ 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "2.6", 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: 5, 2024-09-14T22:55:36.444 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.0", 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: 6, 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: 7 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.2", 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:55:36.445 INFO:teuthology.orchestra.run.smithi076.stdout: 4, 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: 5 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.7", 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: 7, 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.8", 2024-09-14T22:55:36.446 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: 3, 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: 6 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_items": [ 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.3", 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "mappings": [ 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "from": 4, 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: "to": 2 2024-09-14T22:55:36.447 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.12", 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "mappings": [ 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "from": 0, 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "to": 7 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_upmap_primaries": [], 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: "pg_temp": [ 2024-09-14T22:55:36.448 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "pgid": "3.1a", 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "osds": [ 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: 1, 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: 2 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "primary_temp": [], 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklist": {}, 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "range_blocklist": {}, 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "erasure_code_profiles": { 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "default": { 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "crush-failure-domain": "osd", 2024-09-14T22:55:36.449 INFO:teuthology.orchestra.run.smithi076.stdout: "k": "2", 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "m": "1", 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "plugin": "jerasure", 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "technique": "reed_sol_van" 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "removed_snaps_queue": [ 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "snaps": [ 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "begin": 3, 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: "length": 33 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.450 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "new_removed_snaps": [ 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "pool": 3, 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "snaps": [ 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "begin": 3, 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "length": 1 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: ], 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "new_purged_snaps": [], 2024-09-14T22:55:36.451 INFO:teuthology.orchestra.run.smithi076.stdout: "crush_node_flags": {}, 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "device_class_flags": {}, 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode": { 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_enabled": false, 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_bucket_count": 0, 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "degraded_stretch_mode": 0, 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "recovering_stretch_mode": 0, 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: "stretch_mode_bucket": 0 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.452 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:36.454 INFO:tasks.thrashosds.thrasher:j is {'epoch': 162, 'fsid': '9a42a40e-8d25-4801-9835-f823a294bdbe', 'created': '2024-09-14T22:52:37.944451+0000', 'modified': '2024-09-14T22:55:36.050376+0000', 'last_up_change': '2024-09-14T22:54:57.788789+0000', 'last_in_change': '2024-09-14T22:55:31.022383+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T22:52:50.813186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T22:52:55.912547+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T22:53:05.643169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 162, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 0.8300666809082031, 'average_primary_affinity': 0.8299999833106995, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '45f61e20-37e6-4a13-b005-1f135f5e5a58', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5721282958984375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 157, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6800', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6801', 'nonce': 2488220599}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6802', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6803', 'nonce': 2488220599}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6806', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6807', 'nonce': 2488220599}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6804', 'nonce': 2488220599}, {'type': 'v1', 'addr': '172.21.15.76:6805', 'nonce': 2488220599}]}, 'public_addr': '172.21.15.76:6801/2488220599', 'cluster_addr': '172.21.15.76:6803/2488220599', 'heartbeat_back_addr': '172.21.15.76:6807/2488220599', 'heartbeat_front_addr': '172.21.15.76:6805/2488220599', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bc72773d-098f-4ac5-89b5-13dae87b65b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.0684051513671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 158, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6808', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6809', 'nonce': 522829438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6810', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6811', 'nonce': 522829438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6814', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6815', 'nonce': 522829438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6812', 'nonce': 522829438}, {'type': 'v1', 'addr': '172.21.15.76:6813', 'nonce': 522829438}]}, 'public_addr': '172.21.15.76:6809/522829438', 'cluster_addr': '172.21.15.76:6811/522829438', 'heartbeat_back_addr': '172.21.15.76:6815/522829438', 'heartbeat_front_addr': '172.21.15.76:6813/522829438', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2c2a50b0-e6ab-4593-9a26-badf1cc3dd34', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 160, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6824', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6825', 'nonce': 3214755929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6826', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6827', 'nonce': 3214755929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6830', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6831', 'nonce': 3214755929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6828', 'nonce': 3214755929}, {'type': 'v1', 'addr': '172.21.15.76:6829', 'nonce': 3214755929}]}, 'public_addr': '172.21.15.76:6825/3214755929', 'cluster_addr': '172.21.15.76:6827/3214755929', 'heartbeat_back_addr': '172.21.15.76:6831/3214755929', 'heartbeat_front_addr': '172.21.15.76:6829/3214755929', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '40d20d5f-5509-4a9f-8b13-c9e8da7339bc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 157, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6816', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6817', 'nonce': 2699236014}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6818', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6819', 'nonce': 2699236014}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6822', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6823', 'nonce': 2699236014}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6820', 'nonce': 2699236014}, {'type': 'v1', 'addr': '172.21.15.76:6821', 'nonce': 2699236014}]}, 'public_addr': '172.21.15.76:6817/2699236014', 'cluster_addr': '172.21.15.76:6819/2699236014', 'heartbeat_back_addr': '172.21.15.76:6823/2699236014', 'heartbeat_front_addr': '172.21.15.76:6821/2699236014', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '78bf5206-fbf7-4668-a27f-9eb867edc4e5', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 124, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6824', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6825', 'nonce': 2097675638}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6826', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6827', 'nonce': 2097675638}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6830', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6831', 'nonce': 2097675638}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6828', 'nonce': 2097675638}, {'type': 'v1', 'addr': '172.21.15.114:6829', 'nonce': 2097675638}]}, 'public_addr': '172.21.15.114:6825/2097675638', 'cluster_addr': '172.21.15.114:6827/2097675638', 'heartbeat_back_addr': '172.21.15.114:6831/2097675638', 'heartbeat_front_addr': '172.21.15.114:6829/2097675638', 'state': ['exists']}, {'osd': 5, 'uuid': '72838af1-36f2-4cf6-a5ee-6daf5d4ea70a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 157, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6800', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6801', 'nonce': 3469663218}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6802', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6803', 'nonce': 3469663218}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6806', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6807', 'nonce': 3469663218}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6804', 'nonce': 3469663218}, {'type': 'v1', 'addr': '172.21.15.114:6805', 'nonce': 3469663218}]}, 'public_addr': '172.21.15.114:6801/3469663218', 'cluster_addr': '172.21.15.114:6803/3469663218', 'heartbeat_back_addr': '172.21.15.114:6807/3469663218', 'heartbeat_front_addr': '172.21.15.114:6805/3469663218', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '11be725a-e81f-4b8a-8a68-acb54e67b443', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 124, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6816', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6817', 'nonce': 2551471767}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6818', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6819', 'nonce': 2551471767}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6822', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6823', 'nonce': 2551471767}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6820', 'nonce': 2551471767}, {'type': 'v1', 'addr': '172.21.15.114:6821', 'nonce': 2551471767}]}, 'public_addr': '172.21.15.114:6817/2551471767', 'cluster_addr': '172.21.15.114:6819/2551471767', 'heartbeat_back_addr': '172.21.15.114:6823/2551471767', 'heartbeat_front_addr': '172.21.15.114:6821/2551471767', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '94103011-716e-495f-aeb3-759071ce6f14', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 157, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6808', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6809', 'nonce': 941014947}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6810', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6811', 'nonce': 941014947}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6814', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6815', 'nonce': 941014947}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6812', 'nonce': 941014947}, {'type': 'v1', 'addr': '172.21.15.114:6813', 'nonce': 941014947}]}, 'public_addr': '172.21.15.114:6809/941014947', 'cluster_addr': '172.21.15.114:6811/941014947', 'heartbeat_back_addr': '172.21.15.114:6815/941014947', 'heartbeat_front_addr': '172.21.15.114:6813/941014947', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.133153+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.415439+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.880448+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.879405+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:47.079602+0000', 'dead_epoch': 124}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.493580+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.693553+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T22:52:46.442044+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [5, 2]}, {'pgid': '3.0', 'osds': [6, 7]}, {'pgid': '3.2', 'osds': [4, 5]}, {'pgid': '3.7', 'osds': [7, 6]}, {'pgid': '3.8', 'osds': [3, 6]}], 'pg_upmap_items': [{'pgid': '3.3', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.12', 'mappings': [{'from': 0, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.1a', 'osds': [1, 2]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 3, 'length': 33}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 3, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T22:55:36.454 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:55:36.455 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:36.455 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:36.455 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:36.456 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T22:55:36.602 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:36.602 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:36.602 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:36.602 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:36.602 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:36.611 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T22:55:36.717 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:77787 3.4 e40/24 3:22c04624:::rbd_data.1135aa5cee52.0000000000000000:head v 40'103, mlcod=40'101)", 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 129.50694939100001, 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.018524783, 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.718 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208521+0000", 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.208523+0000", 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9080000000000002e-06 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.719 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211684+0000", 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0031605420000000001 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211705+0000", 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1132e-05 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211734+0000", 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.8534000000000002e-05 2024-09-14T22:55:36.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211808+0000", 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4113e-05 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.211828+0000", 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0013e-05 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227006+0000", 2024-09-14T22:55:36.721 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.015178305 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:27.227046+0000", 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.0235999999999999e-05 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:130937 3.5 e52/47 3:a2124333:::rbd_data.1135a9fbf9fc.0000000000000004:head v 52'119, mlcod=0'0)", 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 116.38328761699999, 2024-09-14T22:55:36.722 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0081267300000000004, 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.723 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.332183+0000", 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3099999999999999e-07 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334966+0000", 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027826080000000002 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334979+0000", 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2998e-05 2024-09-14T22:55:36.724 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.334988+0000", 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0399999999999998e-06 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335046+0000", 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.8267000000000003e-05 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.335064+0000", 2024-09-14T22:55:36.725 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7702000000000001e-05 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340260+0000", 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051955480000000004 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:40.340310+0000", 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0336000000000001e-05 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.726 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:136965 3.5 e60/47 3:a74ab5d6:::rbd_data.1135e78a2adf.0000000000000004:head v 60'136, mlcod=60'134)", 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 107.45152014600001, 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0065466450000000002, 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:36.727 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263951+0000", 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.728 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.263952+0000", 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1289999999999999e-06 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266841+0000", 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.002889665 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266857+0000", 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.6016999999999999e-05 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.729 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266865+0000", 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3640000000000001e-06 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266922+0000", 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.7250000000000002e-05 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.266943+0000", 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1350000000000001e-05 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.730 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270447+0000", 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.003503908 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:49.270497+0000", 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9962e-05 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_repop(client.4405.0:172284 3.9 e82/24 3:9139de08:::rbd_data.113598f99d1a.000000000000001d:head v 82'234, mlcod=81'225)", 2024-09-14T22:55:36.731 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 82.541573904000003, 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005213245, 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.732 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173897+0000", 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173898+0000", 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1299999999999999e-07 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173904+0000", 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.733 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173917+0000", 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.2952999999999999e-05 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173923+0000", 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4910000000000002e-06 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173976+0000", 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2927999999999999e-05 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.734 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.173997+0000", 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0835999999999998e-05 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179074+0000", 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050775899999999999 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:14.179110+0000", 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.5664999999999998e-05 2024-09-14T22:55:36.735 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193906 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.136836384000006, 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052257099999999997, 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.736 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193906 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578634+0000", 2024-09-14T22:55:36.737 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578635+0000", 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.13e-07 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578638+0000", 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5770000000000001e-06 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578647+0000", 2024-09-14T22:55:36.738 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0450000000000006e-06 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578653+0000", 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1959999999999996e-06 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583051+0000", 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0043986240000000003 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.739 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583169+0000", 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000118014 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583860+0000", 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00069074100000000003 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193907 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 180224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:36.740 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.136809061999998, 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0058562529999999996, 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193907 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.741 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578662+0000", 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.3099999999999998e-07 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.742 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578664+0000", 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1909999999999999e-06 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578673+0000", 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4360000000000002e-06 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.578678+0000", 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4550000000000003e-06 2024-09-14T22:55:36.743 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.583909+0000", 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0052307270000000001 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584038+0000", 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000128854 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.744 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584518+0000", 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00048015900000000002 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "MOSDPGPushReply(3.1e 94/93 [PushReplyOp(3:7ece0dcb:::rbd_object_map.113518e86835:head)])", 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.136397571000003, 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014560020999999999, 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.745 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579073+0000", 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.746 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579074+0000", 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0160000000000001e-06 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579080+0000", 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1990000000000003e-06 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579097+0000", 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.624e-05 2024-09-14T22:55:36.747 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.579105+0000", 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3620000000000006e-06 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593416+0000", 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.014310666 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593633+0000", 2024-09-14T22:55:36.748 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000217538 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:193956 3.1e 3:79aaf191:::rbd_data.11358204802c.0000000000000018:27 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.134738303999995, 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0053143510000000001, 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.749 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 193956 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.750 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580733+0000", 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4999999999999998e-07 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580737+0000", 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7890000000000001e-06 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580743+0000", 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1099999999999999e-06 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.751 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.580747+0000", 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6249999999999998e-06 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585774+0000", 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0050268589999999998 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.585901+0000", 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000127133 2024-09-14T22:55:36.752 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586047+0000", 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00014538499999999999 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194086 3.1e 3:7c01a02e:::rbd_data.11356c4af504.0000000000000008:25 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.130706880000005, 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054081750000000003, 2024-09-14T22:55:36.753 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194086 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.754 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584764+0000", 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6400000000000003e-07 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584767+0000", 2024-09-14T22:55:36.755 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.379e-06 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584773+0000", 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5379999999999999e-06 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.584782+0000", 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6619999999999994e-06 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.756 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.589901+0000", 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051189759999999999 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590011+0000", 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010967 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.590172+0000", 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00016148599999999999 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.757 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194120 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.128863404000001, 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056311110000000003, 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.758 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194120 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586607+0000", 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.759 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586608+0000", 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9799999999999999e-07 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586611+0000", 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.1140000000000002e-06 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586617+0000", 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9869999999999996e-06 2024-09-14T22:55:36.760 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586622+0000", 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8940000000000001e-06 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591792+0000", 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0051700920000000003 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.591929+0000", 2024-09-14T22:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013729899999999999 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592239+0000", 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030932699999999997 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194124 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 278528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.12884373, 2024-09-14T22:55:36.762 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059298550000000004, 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194124 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:36.763 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586627+0000", 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.65e-07 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.764 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586646+0000", 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8433000000000001e-05 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586653+0000", 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9079999999999998e-06 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586658+0000", 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.091e-06 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.765 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592272+0000", 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0056143199999999999 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592343+0000", 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1247000000000004e-05 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592557+0000", 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00021349100000000001 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.766 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194125 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.128805076000006, 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063143399999999999, 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.767 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194125 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586666+0000", 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.34e-07 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586668+0000", 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586674+0000", 2024-09-14T22:55:36.769 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1090000000000003e-06 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586679+0000", 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8389999999999997e-06 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592588+0000", 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0059088589999999998 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.770 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592673+0000", 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4789000000000004e-05 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.592980+0000", 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00030713000000000002 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:194129 3.1e 3:784665b0:::rbd_data.113598f99d1a.0000000000000003:23 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94)", 2024-09-14T22:55:36.771 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 70.128785195999995, 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0067018640000000001, 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 194129 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.772 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586686+0000", 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4099999999999999e-07 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.773 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586688+0000", 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9640000000000002e-06 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586694+0000", 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1140000000000002e-06 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.586699+0000", 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8400000000000002e-06 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593020+0000", 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0063214279999999996 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593095+0000", 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4375999999999994e-05 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:26.593387+0000", 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00029270099999999997 2024-09-14T22:55:36.775 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197008 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 65.755614003000005, 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.005409105, 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.776 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197008 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.777 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959857+0000", 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2300000000000001e-07 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959861+0000", 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7469999999999999e-06 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959870+0000", 2024-09-14T22:55:36.778 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.456e-06 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.959876+0000", 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0179999999999998e-06 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960594+0000", 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00071825899999999995 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.779 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960643+0000", 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9203999999999998e-05 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00023453800000000001 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965053+0000", 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0041749780000000002 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.780 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965162+0000", 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000109479 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965222+0000", 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9539e-05 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.965266+0000", 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4264000000000002e-05 2024-09-14T22:55:36.781 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197011 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 65.754612709, 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0069825599999999996, 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.782 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197011 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960858+0000", 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.783 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960859+0000", 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7100000000000011e-07 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960864+0000", 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3959999999999997e-06 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960878+0000", 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3760999999999999e-05 2024-09-14T22:55:36.784 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.960886+0000", 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8949999999999998e-06 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961916+0000", 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0010294880000000001 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.785 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.961995+0000", 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9598999999999995e-05 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.962245+0000", 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000249886 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967697+0000", 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0054518830000000002 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.786 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967747+0000", 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0065000000000003e-05 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967803+0000", 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.6347000000000001e-05 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:30.967841+0000", 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7268999999999998e-05 2024-09-14T22:55:36.787 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:197509 3.4 3:26a81f63:::rbd_object_map.1135683a1f7c:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 64.448556518000004, 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0071898780000000002, 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.788 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 197509 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266914+0000", 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.789 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266915+0000", 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.096e-06 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266922+0000", 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3720000000000001e-06 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.790 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266936+0000", 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.4506e-05 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.266944+0000", 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.8250000000000005e-06 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268652+0000", 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0017076979999999999 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.791 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.268735+0000", 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3265999999999999e-05 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.269114+0000", 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00037922200000000002 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.273877+0000", 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0047623240000000001 2024-09-14T22:55:36.792 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274011+0000", 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00013423299999999999 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274071+0000", 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0334000000000002e-05 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.793 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:32.274104+0000", 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.3002e-05 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:211119 3.4 3:26977db1:::rbd_data.11354571ccdf.0000000000000332:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e126)", 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 36.831837078, 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0055643710000000002, 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.794 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 211119 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.795 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883634+0000", 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883635+0000", 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.032e-06 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883640+0000", 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1649999999999997e-06 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.796 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883653+0000", 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3241000000000001e-05 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883661+0000", 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7389999999999999e-06 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.797 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883723+0000", 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2403000000000005e-05 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.883845+0000", 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00012148 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 1", 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.884653+0000", 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00080859200000000001 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.798 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.886346+0000", 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.001692268 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889139+0000", 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0027930920000000001 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889171+0000", 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2595999999999998e-05 2024-09-14T22:55:36.799 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:54:59.889198+0000", 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6763000000000001e-05 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:213874 3.3 3:c31b501c:::rbd_header.1135a4365f9c:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 19.776758820000001, 2024-09-14T22:55:36.800 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0068492850000000001, 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 213874 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.801 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938712+0000", 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938714+0000", 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1059999999999998e-06 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.802 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938723+0000", 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.0180000000000007e-06 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938745+0000", 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1985e-05 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938755+0000", 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0051000000000001e-05 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.803 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938843+0000", 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.7671999999999997e-05 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op must wait for map", 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.938849+0000", 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.6080000000000001e-06 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941261+0000", 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0024116459999999999 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.804 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941361+0000", 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.9846999999999995e-05 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.941720+0000", 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00035947399999999998 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "op_commit", 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945393+0000", 2024-09-14T22:55:36.805 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.0036722679999999998 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945439+0000", 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6186000000000001e-05 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "commit_sent", 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945507+0000", 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7947999999999994e-05 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.806 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:16.945561+0000", 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4475999999999998e-05 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:214566 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e148)", 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 14.730230174999999, 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.98538225400000001, 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:36.807 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 214566 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.808 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985241+0000", 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985243+0000", 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.244e-06 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985254+0000", 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1428e-05 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.809 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985275+0000", 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0727e-05 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985284+0000", 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.2420000000000006e-06 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985354+0000", 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9548000000000006e-05 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.810 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:21.985363+0000", 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.1109999999999997e-06 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970284+0000", 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.98492126999999996 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970438+0000", 2024-09-14T22:55:36.811 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00015336200000000001 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:22.970623+0000", 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000185322 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:216079 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:55:31.045493+0000", 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 5.6699780740000003, 2024-09-14T22:55:36.812 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.99245719600000004, 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 216079 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:31.045493+0000", 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.813 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:31.045493+0000", 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:31.045495+0000", 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3420000000000002e-06 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:31.045507+0000", 2024-09-14T22:55:36.814 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.167e-05 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:31.045526+0000", 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.9199999999999999e-05 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:31.045536+0000", 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0117e-05 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.815 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:31.045618+0000", 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.1934999999999994e-05 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:31.045632+0000", 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.3929e-05 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:32.037933+0000", 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.99230094899999999 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:36.816 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:55:32.037950+0000", 2024-09-14T22:55:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7054e-05 2024-09-14T22:55:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:36.817 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:36.817 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:36.818 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T22:55:36.819 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:55:36.820 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:55:36.821 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-14T22:55:36.055102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":368422019,"num_objects":1671,"num_object_clones":129,"num_object_copies":3342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":786,"num_objects_misplaced":46,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":1506,"num_read":260522,"num_read_kb":1066494,"num_write":19082,"num_write_kb":705292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1535,"num_bytes_recovered":141629863,"num_keys_recovered":114,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18584,"ondisk_log_size":18584,"up":69,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":565456,"kb_used_data":214604,"kb_used_omap":12,"kb_used_meta":350643,"kb_avail":754409264,"statfs":{"total":773094113280,"available":772515086336,"internally_reserved":0,"allocated":219754496,"data_stored":233510923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":359058993},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-38251417,"num_objects":-71,"num_object_clones":-15,"num_object_copies":-142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-10,"num_objects_misplaced":-101,"num_objects_unfound":0,"num_objects_dirty":-71,"num_whiteouts":-10,"num_read":684,"num_read_kb":585,"num_write":148,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":71244,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022559"},"pg_stats":[{"pgid":"3.1f","version":"159'678","reported_seq":8356,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.046286+0000","last_change":"2024-09-14T22:55:34.047008+0000","last_active":"2024-09-14T22:55:34.046286+0000","last_peered":"2024-09-14T22:55:34.046286+0000","last_clean":"2024-09-14T22:55:34.046286+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:55:34.046286+0000","last_undegraded":"2024-09-14T22:55:34.046286+0000","last_fullsized":"2024-09-14T22:55:34.046286+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:12:43.392474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012416440000000001,"stat_sum":{"num_bytes":16,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":38,"num_read":9379,"num_read_kb":31309,"num_write":713,"num_write_kb":7692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.1e","version":"160'529","reported_seq":8551,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.050556+0000","last_change":"2024-09-14T22:55:34.050910+0000","last_active":"2024-09-14T22:55:34.050556+0000","last_peered":"2024-09-14T22:55:34.050556+0000","last_clean":"2024-09-14T22:55:34.050556+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:55:34.050556+0000","last_undegraded":"2024-09-14T22:55:34.050556+0000","last_fullsized":"2024-09-14T22:55:34.050556+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:01:32.852702+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.004642523,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8409,"num_read_kb":20232,"num_write":463,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.1d","version":"155'493","reported_seq":9031,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.046129+0000","last_change":"2024-09-14T22:54:58.799496+0000","last_active":"2024-09-14T22:55:34.046129+0000","last_peered":"2024-09-14T22:55:34.046129+0000","last_clean":"2024-09-14T22:54:56.880591+0000","last_became_active":"2024-09-14T22:54:58.799496+0000","last_became_peered":"2024-09-14T22:54:58.799496+0000","last_unstale":"2024-09-14T22:55:34.046129+0000","last_undegraded":"2024-09-14T22:54:58.798355+0000","last_fullsized":"2024-09-14T22:54:58.798271+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:12:38.616281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000895684,"stat_sum":{"num_bytes":52771974,"num_objects":73,"num_object_clones":14,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":73,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":59,"num_read":9221,"num_read_kb":38644,"num_write":441,"num_write_kb":52016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":73}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"160'631","reported_seq":3373,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.095358+0000","last_change":"2024-09-14T22:54:58.799448+0000","last_active":"2024-09-14T22:55:34.095358+0000","last_peered":"2024-09-14T22:55:34.095358+0000","last_clean":"2024-09-14T22:54:56.910976+0000","last_became_active":"2024-09-14T22:54:58.799448+0000","last_became_peered":"2024-09-14T22:54:58.799448+0000","last_unstale":"2024-09-14T22:55:34.095358+0000","last_undegraded":"2024-09-14T22:54:58.798221+0000","last_fullsized":"2024-09-14T22:54:58.798125+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:58:11.230576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010762059999999999,"stat_sum":{"num_bytes":18143866,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":41,"num_read":4483,"num_read_kb":12173,"num_write":712,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":51}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"156'322","reported_seq":5128,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.045730+0000","last_change":"2024-09-14T22:54:58.804209+0000","last_active":"2024-09-14T22:55:34.045730+0000","last_peered":"2024-09-14T22:55:34.045730+0000","last_clean":"2024-09-14T22:54:56.847408+0000","last_became_active":"2024-09-14T22:54:58.804209+0000","last_became_peered":"2024-09-14T22:54:58.804209+0000","last_unstale":"2024-09-14T22:55:34.045730+0000","last_undegraded":"2024-09-14T22:54:58.803622+0000","last_fullsized":"2024-09-14T22:54:58.803532+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:00:44.726682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031494579999999999,"stat_sum":{"num_bytes":31144944,"num_objects":59,"num_object_clones":8,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":59,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":49,"num_read":4616,"num_read_kb":54005,"num_write":310,"num_write_kb":18747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":59}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"155'691","reported_seq":14473,"reported_epoch":160,"state":"active+clean+remapped","last_fresh":"2024-09-14T22:55:34.134087+0000","last_change":"2024-09-14T22:55:34.051269+0000","last_active":"2024-09-14T22:55:34.134087+0000","last_peered":"2024-09-14T22:55:34.134087+0000","last_clean":"2024-09-14T22:55:34.134087+0000","last_became_active":"2024-09-14T22:55:33.045904+0000","last_became_peered":"2024-09-14T22:55:33.045904+0000","last_unstale":"2024-09-14T22:55:34.134087+0000","last_undegraded":"2024-09-14T22:55:34.134087+0000","last_fullsized":"2024-09-14T22:55:34.134087+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:03:54.491745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082869300000000003,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":16199,"num_read_kb":53489,"num_write":537,"num_write_kb":29099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":18306298,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":46}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.19","version":"160'485","reported_seq":16147,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.115463+0000","last_change":"2024-09-14T22:55:34.052132+0000","last_active":"2024-09-14T22:55:34.115463+0000","last_peered":"2024-09-14T22:55:34.115463+0000","last_clean":"2024-09-14T22:55:34.115463+0000","last_became_active":"2024-09-14T22:55:32.040701+0000","last_became_peered":"2024-09-14T22:55:32.040701+0000","last_unstale":"2024-09-14T22:55:34.115463+0000","last_undegraded":"2024-09-14T22:55:34.115463+0000","last_fullsized":"2024-09-14T22:55:34.115463+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:11:50.047894+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059748170000000003,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":15953,"num_read_kb":92561,"num_write":414,"num_write_kb":38470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.18","version":"155'674","reported_seq":2381,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.046369+0000","last_change":"2024-09-14T22:55:34.047215+0000","last_active":"2024-09-14T22:55:34.046369+0000","last_peered":"2024-09-14T22:55:34.046369+0000","last_clean":"2024-09-14T22:55:34.046369+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:55:34.046369+0000","last_undegraded":"2024-09-14T22:55:34.046369+0000","last_fullsized":"2024-09-14T22:55:34.046369+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:04:13.947659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010220419999999999,"stat_sum":{"num_bytes":786432,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":51,"num_read":2959,"num_read_kb":22740,"num_write":860,"num_write_kb":10481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.17","version":"160'635","reported_seq":6816,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.148898+0000","last_change":"2024-09-14T22:55:34.046661+0000","last_active":"2024-09-14T22:55:34.148898+0000","last_peered":"2024-09-14T22:55:34.148898+0000","last_clean":"2024-09-14T22:55:34.148898+0000","last_became_active":"2024-09-14T22:53:38.881857+0000","last_became_peered":"2024-09-14T22:53:38.881857+0000","last_unstale":"2024-09-14T22:55:34.148898+0000","last_undegraded":"2024-09-14T22:55:34.148898+0000","last_fullsized":"2024-09-14T22:55:34.148898+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:53:38.636232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069061399999999998,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":37,"num_read":7774,"num_read_kb":34710,"num_write":806,"num_write_kb":27726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.16","version":"159'432","reported_seq":9712,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.046802+0000","last_change":"2024-09-14T22:55:34.046802+0000","last_active":"2024-09-14T22:55:34.046802+0000","last_peered":"2024-09-14T22:55:34.046802+0000","last_clean":"2024-09-14T22:55:34.046802+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:55:34.046802+0000","last_undegraded":"2024-09-14T22:55:34.046802+0000","last_fullsized":"2024-09-14T22:55:34.046802+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:57:04.884086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060923900000000003,"stat_sum":{"num_bytes":786432,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":39,"num_read":10352,"num_read_kb":46008,"num_write":346,"num_write_kb":32677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.15","version":"160'420","reported_seq":9355,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.097074+0000","last_change":"2024-09-14T22:55:34.097074+0000","last_active":"2024-09-14T22:55:34.097074+0000","last_peered":"2024-09-14T22:55:34.097074+0000","last_clean":"2024-09-14T22:55:34.097074+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:55:34.097074+0000","last_undegraded":"2024-09-14T22:55:34.097074+0000","last_fullsized":"2024-09-14T22:55:34.097074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:42:26.060508+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0019647699999999998,"stat_sum":{"num_bytes":993212,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":39,"num_read":10004,"num_read_kb":43389,"num_write":401,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.14","version":"159'567","reported_seq":10934,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.094869+0000","last_change":"2024-09-14T22:54:58.806822+0000","last_active":"2024-09-14T22:55:34.094869+0000","last_peered":"2024-09-14T22:55:34.094869+0000","last_clean":"2024-09-14T22:54:56.923601+0000","last_became_active":"2024-09-14T22:54:58.806822+0000","last_became_peered":"2024-09-14T22:54:58.806822+0000","last_unstale":"2024-09-14T22:55:34.094869+0000","last_undegraded":"2024-09-14T22:54:58.805967+0000","last_fullsized":"2024-09-14T22:54:58.805881+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:38:48.931307+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0071227479999999999,"stat_sum":{"num_bytes":46430735,"num_objects":77,"num_object_clones":16,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":77,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":59,"num_read":11380,"num_read_kb":66402,"num_write":621,"num_write_kb":37521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":77}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":296,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.049075+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:55:34.049075+0000","last_peered":"2024-09-14T22:55:34.049075+0000","last_clean":"2024-09-14T22:55:34.049075+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:55:34.049075+0000","last_undegraded":"2024-09-14T22:55:34.049075+0000","last_fullsized":"2024-09-14T22:55:34.049075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"160'714","reported_seq":7980,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.049499+0000","last_change":"2024-09-14T22:55:34.049499+0000","last_active":"2024-09-14T22:55:34.049499+0000","last_peered":"2024-09-14T22:55:34.049499+0000","last_clean":"2024-09-14T22:55:34.049499+0000","last_became_active":"2024-09-14T22:55:32.040243+0000","last_became_peered":"2024-09-14T22:55:32.040243+0000","last_unstale":"2024-09-14T22:55:34.049499+0000","last_undegraded":"2024-09-14T22:55:34.049499+0000","last_fullsized":"2024-09-14T22:55:34.049499+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:38:37.986783+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0035839190000000001,"stat_sum":{"num_bytes":47,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":36,"num_read":9390,"num_read_kb":42175,"num_write":737,"num_write_kb":24662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"2.6","version":"0'0","reported_seq":255,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.045899+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:55:34.045899+0000","last_peered":"2024-09-14T22:55:34.045899+0000","last_clean":"2024-09-14T22:55:34.045899+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:55:34.045899+0000","last_undegraded":"2024-09-14T22:55:34.045899+0000","last_fullsized":"2024-09-14T22:55:34.045899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"156'335","reported_seq":2059,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.097127+0000","last_change":"2024-09-14T22:55:34.097549+0000","last_active":"2024-09-14T22:55:34.097127+0000","last_peered":"2024-09-14T22:55:34.097127+0000","last_clean":"2024-09-14T22:55:34.097127+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:55:34.097127+0000","last_undegraded":"2024-09-14T22:55:34.097127+0000","last_fullsized":"2024-09-14T22:55:34.097127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:34:33.558938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002227588,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":1234,"num_read_kb":15244,"num_write":279,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"2.5","version":"0'0","reported_seq":296,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.095240+0000","last_change":"2024-09-14T22:53:00.415656+0000","last_active":"2024-09-14T22:55:34.095240+0000","last_peered":"2024-09-14T22:55:34.095240+0000","last_clean":"2024-09-14T22:55:34.095240+0000","last_became_active":"2024-09-14T22:52:57.528873+0000","last_became_peered":"2024-09-14T22:52:57.528873+0000","last_unstale":"2024-09-14T22:55:34.095240+0000","last_undegraded":"2024-09-14T22:55:34.095240+0000","last_fullsized":"2024-09-14T22:55:34.095240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:16:04.852595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"159'841","reported_seq":14247,"reported_epoch":159,"state":"peering","last_fresh":"2024-09-14T22:55:33.245454+0000","last_change":"2024-09-14T22:55:34.049719+0000","last_active":"2024-09-14T22:55:34.049719+0000","last_peered":"2024-09-14T22:55:33.245454+0000","last_clean":"2024-09-14T22:55:30.133615+0000","last_became_active":"2024-09-14T22:55:33.059500+0000","last_became_peered":"2024-09-14T22:55:33.059500+0000","last_unstale":"2024-09-14T22:55:33.245454+0000","last_undegraded":"2024-09-14T22:55:33.245454+0000","last_fullsized":"2024-09-14T22:55:33.045026+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060501399999999996,"stat_sum":{"num_bytes":2121636,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":45,"num_read":15582,"num_read_kb":40098,"num_write":1078,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":15786495,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"1d"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.050838+0000","last_change":"2024-09-14T22:55:32.039844+0000","last_active":"2024-09-14T22:55:34.050838+0000","last_peered":"2024-09-14T22:55:34.050838+0000","last_clean":"2024-09-14T22:55:34.050838+0000","last_became_active":"2024-09-14T22:55:32.039508+0000","last_became_peered":"2024-09-14T22:55:32.039508+0000","last_unstale":"2024-09-14T22:55:34.050838+0000","last_undegraded":"2024-09-14T22:55:34.050838+0000","last_fullsized":"2024-09-14T22:55:34.050838+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:04.853912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"160'432","reported_seq":15723,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.050584+0000","last_change":"2024-09-14T22:55:34.050898+0000","last_active":"2024-09-14T22:55:34.050584+0000","last_peered":"2024-09-14T22:55:34.050584+0000","last_clean":"2024-09-14T22:55:34.050584+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:55:34.050584+0000","last_undegraded":"2024-09-14T22:55:34.050584+0000","last_fullsized":"2024-09-14T22:55:34.050584+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:55:24.886006+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0044715509999999998,"stat_sum":{"num_bytes":1572864,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":41,"num_read":14507,"num_read_kb":48865,"num_write":355,"num_write_kb":19125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.3","version":"160'993","reported_seq":6759,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.120667+0000","last_change":"2024-09-14T22:55:34.054085+0000","last_active":"2024-09-14T22:55:34.120667+0000","last_peered":"2024-09-14T22:55:34.120667+0000","last_clean":"2024-09-14T22:55:34.120667+0000","last_became_active":"2024-09-14T22:54:47.747387+0000","last_became_peered":"2024-09-14T22:54:47.747387+0000","last_unstale":"2024-09-14T22:55:34.120667+0000","last_undegraded":"2024-09-14T22:55:34.120667+0000","last_fullsized":"2024-09-14T22:55:34.120667+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":993,"log_dups_size":0,"ondisk_log_size":993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:07.315698+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0053814010000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":9059,"num_read_kb":57930,"num_write":1033,"num_write_kb":41146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":29933802,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"2.2","version":"22'2","reported_seq":316,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.045796+0000","last_change":"2024-09-14T22:55:32.043590+0000","last_active":"2024-09-14T22:55:34.045796+0000","last_peered":"2024-09-14T22:55:34.045796+0000","last_clean":"2024-09-14T22:55:34.045796+0000","last_became_active":"2024-09-14T22:55:32.040648+0000","last_became_peered":"2024-09-14T22:55:32.040648+0000","last_unstale":"2024-09-14T22:55:34.045796+0000","last_undegraded":"2024-09-14T22:55:34.045796+0000","last_fullsized":"2024-09-14T22:55:34.045796+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:45:24.150360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"160'369","reported_seq":1841,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.050302+0000","last_change":"2024-09-14T22:55:34.050302+0000","last_active":"2024-09-14T22:55:34.050302+0000","last_peered":"2024-09-14T22:55:34.050302+0000","last_clean":"2024-09-14T22:55:34.050302+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:55:34.050302+0000","last_undegraded":"2024-09-14T22:55:34.050302+0000","last_fullsized":"2024-09-14T22:55:34.050302+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:33:29.537024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001080936,"stat_sum":{"num_bytes":842779,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":1390,"num_read_kb":9489,"num_write":335,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"2.1","version":"0'0","reported_seq":306,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.048408+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:55:34.048408+0000","last_peered":"2024-09-14T22:55:34.048408+0000","last_clean":"2024-09-14T22:55:34.048408+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:55:34.048408+0000","last_undegraded":"2024-09-14T22:55:34.048408+0000","last_fullsized":"2024-09-14T22:55:34.048408+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"159'616","reported_seq":9149,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.045636+0000","last_change":"2024-09-14T22:54:58.803980+0000","last_active":"2024-09-14T22:55:34.045636+0000","last_peered":"2024-09-14T22:55:34.045636+0000","last_clean":"2024-09-14T22:54:56.867795+0000","last_became_active":"2024-09-14T22:54:58.803980+0000","last_became_peered":"2024-09-14T22:54:58.803980+0000","last_unstale":"2024-09-14T22:55:34.045636+0000","last_undegraded":"2024-09-14T22:54:58.803560+0000","last_fullsized":"2024-09-14T22:54:58.803473+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:09:30.624105+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0027371499999999998,"stat_sum":{"num_bytes":12033725,"num_objects":57,"num_object_clones":8,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":49,"num_read":10199,"num_read_kb":21422,"num_write":735,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":57}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":306,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.094435+0000","last_change":"2024-09-14T22:55:32.040027+0000","last_active":"2024-09-14T22:55:34.094435+0000","last_peered":"2024-09-14T22:55:34.094435+0000","last_clean":"2024-09-14T22:55:34.094435+0000","last_became_active":"2024-09-14T22:55:32.039728+0000","last_became_peered":"2024-09-14T22:55:32.039728+0000","last_unstale":"2024-09-14T22:55:34.094435+0000","last_undegraded":"2024-09-14T22:55:34.094435+0000","last_fullsized":"2024-09-14T22:55:34.094435+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:36.095183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"159'464","reported_seq":6833,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.123581+0000","last_change":"2024-09-14T22:54:58.798769+0000","last_active":"2024-09-14T22:55:34.123581+0000","last_peered":"2024-09-14T22:55:34.123581+0000","last_clean":"2024-09-14T22:54:56.797201+0000","last_became_active":"2024-09-14T22:54:58.798769+0000","last_became_peered":"2024-09-14T22:54:58.798769+0000","last_unstale":"2024-09-14T22:55:34.123581+0000","last_undegraded":"2024-09-14T22:54:58.798177+0000","last_fullsized":"2024-09-14T22:54:58.798081+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:12:18.194711+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0032582119999999999,"stat_sum":{"num_bytes":48172756,"num_objects":80,"num_object_clones":11,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":80,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":68,"num_read":6223,"num_read_kb":32196,"num_write":425,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":80}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":297,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.045792+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:55:34.045792+0000","last_peered":"2024-09-14T22:55:34.045792+0000","last_clean":"2024-09-14T22:55:34.045792+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:55:34.045792+0000","last_undegraded":"2024-09-14T22:55:34.045792+0000","last_fullsized":"2024-09-14T22:55:34.045792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":447,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.094501+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:55:34.094501+0000","last_peered":"2024-09-14T22:55:34.094501+0000","last_clean":"2024-09-14T22:55:34.094501+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:55:34.094501+0000","last_undegraded":"2024-09-14T22:55:34.094501+0000","last_fullsized":"2024-09-14T22:55:34.094501+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"159'918","reported_seq":10121,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.134428+0000","last_change":"2024-09-14T22:55:34.047285+0000","last_active":"2024-09-14T22:55:34.134428+0000","last_peered":"2024-09-14T22:55:34.134428+0000","last_clean":"2024-09-14T22:55:34.134428+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:55:34.134428+0000","last_undegraded":"2024-09-14T22:55:34.134428+0000","last_fullsized":"2024-09-14T22:55:34.134428+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":918,"log_dups_size":0,"ondisk_log_size":918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:10:03.618402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140832,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":47,"num_read":14356,"num_read_kb":35997,"num_write":931,"num_write_kb":13110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.9","version":"156'505","reported_seq":2284,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.048367+0000","last_change":"2024-09-14T22:54:58.807490+0000","last_active":"2024-09-14T22:55:34.048367+0000","last_peered":"2024-09-14T22:55:34.048367+0000","last_clean":"2024-09-14T22:54:55.796769+0000","last_became_active":"2024-09-14T22:54:58.807490+0000","last_became_peered":"2024-09-14T22:54:58.807490+0000","last_unstale":"2024-09-14T22:55:34.048367+0000","last_undegraded":"2024-09-14T22:54:58.805943+0000","last_fullsized":"2024-09-14T22:54:58.805857+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:32:59.642021+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0045630990000000001,"stat_sum":{"num_bytes":15390567,"num_objects":79,"num_object_clones":13,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":66,"num_read":2706,"num_read_kb":9423,"num_write":453,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":79}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"160'514","reported_seq":2271,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.049219+0000","last_change":"2024-09-14T22:54:58.806678+0000","last_active":"2024-09-14T22:55:34.049219+0000","last_peered":"2024-09-14T22:55:34.049219+0000","last_clean":"2024-09-14T22:54:56.878903+0000","last_became_active":"2024-09-14T22:54:58.806678+0000","last_became_peered":"2024-09-14T22:54:58.806678+0000","last_unstale":"2024-09-14T22:55:34.049219+0000","last_undegraded":"2024-09-14T22:54:58.805953+0000","last_fullsized":"2024-09-14T22:54:58.805833+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:17:45.046209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027641050000000002,"stat_sum":{"num_bytes":12106167,"num_objects":53,"num_object_clones":8,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":44,"num_read":3233,"num_read_kb":8550,"num_write":521,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":53}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'689","reported_seq":4077,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.053674+0000","last_change":"2024-09-14T22:55:34.053934+0000","last_active":"2024-09-14T22:55:34.053674+0000","last_peered":"2024-09-14T22:55:34.053674+0000","last_clean":"2024-09-14T22:55:34.053674+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:55:34.053674+0000","last_undegraded":"2024-09-14T22:55:34.053674+0000","last_fullsized":"2024-09-14T22:55:34.053674+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:02:10.943270+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0075688630000000003,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5954,"num_read_kb":20565,"num_write":618,"num_write_kb":24613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.c","version":"160'576","reported_seq":5684,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.117971+0000","last_change":"2024-09-14T22:55:34.051117+0000","last_active":"2024-09-14T22:55:34.117971+0000","last_peered":"2024-09-14T22:55:34.117971+0000","last_clean":"2024-09-14T22:55:34.117971+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:55:34.117971+0000","last_undegraded":"2024-09-14T22:55:34.117971+0000","last_fullsized":"2024-09-14T22:55:34.117971+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:40:22.982893+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0046659070000000004,"stat_sum":{"num_bytes":47,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":48,"num_read":6241,"num_read_kb":22047,"num_write":552,"num_write_kb":16971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.d","version":"159'623","reported_seq":7248,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.096794+0000","last_change":"2024-09-14T22:55:34.097184+0000","last_active":"2024-09-14T22:55:34.096794+0000","last_peered":"2024-09-14T22:55:34.096794+0000","last_clean":"2024-09-14T22:55:34.096794+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:55:34.096794+0000","last_undegraded":"2024-09-14T22:55:34.096794+0000","last_fullsized":"2024-09-14T22:55:34.096794+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:22:05.172256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020038399999999998,"stat_sum":{"num_bytes":3130654,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":44,"num_read":8194,"num_read_kb":36336,"num_write":767,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.e","version":"155'491","reported_seq":6912,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.095430+0000","last_change":"2024-09-14T22:54:58.807280+0000","last_active":"2024-09-14T22:55:34.095430+0000","last_peered":"2024-09-14T22:55:34.095430+0000","last_clean":"2024-09-14T22:54:56.804068+0000","last_became_active":"2024-09-14T22:54:58.807280+0000","last_became_peered":"2024-09-14T22:54:58.807280+0000","last_unstale":"2024-09-14T22:55:34.095430+0000","last_undegraded":"2024-09-14T22:54:58.806636+0000","last_fullsized":"2024-09-14T22:54:58.806560+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:51:07.822928+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.004453846,"stat_sum":{"num_bytes":43377882,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":63,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":51,"num_read":7727,"num_read_kb":40328,"num_write":448,"num_write_kb":21333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":63}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"159'324","reported_seq":6259,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.095445+0000","last_change":"2024-09-14T22:54:58.807106+0000","last_active":"2024-09-14T22:55:34.095445+0000","last_peered":"2024-09-14T22:55:34.095445+0000","last_clean":"2024-09-14T22:54:56.806754+0000","last_became_active":"2024-09-14T22:54:58.807106+0000","last_became_peered":"2024-09-14T22:54:58.807106+0000","last_unstale":"2024-09-14T22:55:34.095445+0000","last_undegraded":"2024-09-14T22:54:58.806306+0000","last_fullsized":"2024-09-14T22:54:58.806222+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:26:27.999974+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0066796479999999998,"stat_sum":{"num_bytes":16654794,"num_objects":65,"num_object_clones":12,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":65,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":53,"num_read":5750,"num_read_kb":21575,"num_write":279,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":65}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'550","reported_seq":10095,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.049428+0000","last_change":"2024-09-14T22:55:34.050032+0000","last_active":"2024-09-14T22:55:34.049428+0000","last_peered":"2024-09-14T22:55:34.049428+0000","last_clean":"2024-09-14T22:55:34.049428+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:55:34.049428+0000","last_undegraded":"2024-09-14T22:55:34.049428+0000","last_fullsized":"2024-09-14T22:55:34.049428+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:45:03.935723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076665999999999995,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":11616,"num_read_kb":30610,"num_write":481,"num_write_kb":42134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.11","version":"159'588","reported_seq":5464,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.094620+0000","last_change":"2024-09-14T22:54:58.806600+0000","last_active":"2024-09-14T22:55:34.094620+0000","last_peered":"2024-09-14T22:55:34.094620+0000","last_clean":"2024-09-14T22:54:56.804588+0000","last_became_active":"2024-09-14T22:54:58.806600+0000","last_became_peered":"2024-09-14T22:54:58.806600+0000","last_unstale":"2024-09-14T22:55:34.094620+0000","last_undegraded":"2024-09-14T22:54:58.805884+0000","last_fullsized":"2024-09-14T22:54:58.805785+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:37:37.625012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004750013,"stat_sum":{"num_bytes":27901474,"num_objects":61,"num_object_clones":12,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":61,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":48,"num_read":6204,"num_read_kb":34738,"num_write":711,"num_write_kb":24203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":61}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"160'399","reported_seq":1570,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-09-14T22:55:34.146805+0000","last_change":"2024-09-14T22:55:34.096738+0000","last_active":"2024-09-14T22:55:34.146805+0000","last_peered":"2024-09-14T22:55:34.146805+0000","last_clean":"2024-09-14T22:55:34.146805+0000","last_became_active":"2024-09-14T22:53:38.881858+0000","last_became_peered":"2024-09-14T22:53:38.881858+0000","last_unstale":"2024-09-14T22:55:34.146805+0000","last_undegraded":"2024-09-14T22:55:34.146805+0000","last_fullsized":"2024-09-14T22:55:34.146805+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:46:35.002021+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.002207096,"stat_sum":{"num_bytes":917556,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":47,"num_read":1249,"num_read_kb":10584,"num_write":415,"num_write_kb":9648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5","length":"1f"}]},{"pgid":"3.13","version":"159'966","reported_seq":6556,"reported_epoch":160,"state":"active+undersized+degraded","last_fresh":"2024-09-14T22:55:34.094525+0000","last_change":"2024-09-14T22:54:58.808019+0000","last_active":"2024-09-14T22:55:34.094525+0000","last_peered":"2024-09-14T22:55:34.094525+0000","last_clean":"2024-09-14T22:54:56.925007+0000","last_became_active":"2024-09-14T22:54:58.808019+0000","last_became_peered":"2024-09-14T22:54:58.808019+0000","last_unstale":"2024-09-14T22:55:34.094525+0000","last_undegraded":"2024-09-14T22:54:58.806325+0000","last_fullsized":"2024-09-14T22:54:58.806196+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":31,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:55:14.549639+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0070951970000000001,"stat_sum":{"num_bytes":32551054,"num_objects":68,"num_object_clones":8,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":59,"num_read":8863,"num_read_kb":12560,"num_write":1144,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":68}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":367831632,"num_objects":1667,"num_object_clones":129,"num_object_copies":3334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":786,"num_objects_misplaced":46,"num_objects_unfound":0,"num_objects_dirty":1667,"num_whiteouts":1506,"num_read":260406,"num_read_kb":1066394,"num_write":18921,"num_write_kb":702874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":141629825,"num_keys_recovered":114,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202022912,"data_stored":223483962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18464,"ondisk_log_size":18464,"up":51,"acting":52,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84412,"kb_used_data":35160,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94287428,"statfs":{"total":96636764160,"available":96550326272,"internally_reserved":0,"allocated":36003840,"data_stored":34834666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47652,"kb_used_data":3972,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324188,"statfs":{"total":96636764160,"available":96587968512,"internally_reserved":0,"allocated":4067328,"data_stored":3062785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72576,"kb_used_data":23336,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94299264,"statfs":{"total":96636764160,"available":96562446336,"internally_reserved":0,"allocated":23896064,"data_stored":22806415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":185604,"kb_used_data":128048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94186236,"statfs":{"total":96636764160,"available":96446705664,"internally_reserved":0,"allocated":131121152,"data_stored":152203661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":945129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42564,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329276,"statfs":{"total":96636764160,"available":96593178624,"internally_reserved":0,"allocated":3129344,"data_stored":2108901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28976,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342864,"statfs":{"total":96636764160,"available":96607092736,"internally_reserved":0,"allocated":1994752,"data_stored":991338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542182,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70592,"kb_used_data":17192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94301248,"statfs":{"total":96636764160,"available":96564477952,"internally_reserved":0,"allocated":17604608,"data_stored":16558028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14340096,"data_stored":14303995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":16846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1208320,"data_stored":1173991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":10219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":128823296,"data_stored":150670162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21970944,"data_stored":21871486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2146304,"data_stored":2127875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33488896,"data_stored":33309388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:55:36.823 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [7, 6] 2024-09-14T22:55:36.824 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.5', '7', '6'] 2024-09-14T22:55:36.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.5 7 6 2024-09-14T22:55:36.967 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:36.967 INFO:teuthology.orchestra.run.smithi076.stdout: "num_ops": 0 2024-09-14T22:55:36.967 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:36.978 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T22:55:37.073 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:55:37.100 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:37.100 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [], 2024-09-14T22:55:37.100 INFO:teuthology.orchestra.run.smithi076.stdout: "complaint_time": 30, 2024-09-14T22:55:37.100 INFO:teuthology.orchestra.run.smithi076.stdout: "num_blocked_ops": 0 2024-09-14T22:55:37.100 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:37.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T22:55:37.262 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:55:37.262 INFO:teuthology.orchestra.run.smithi076.stdout: "size": 20, 2024-09-14T22:55:37.262 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 600, 2024-09-14T22:55:37.262 INFO:teuthology.orchestra.run.smithi076.stdout: "ops": [ 2024-09-14T22:55:37.262 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.262 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30306 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1374208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.262 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35575030300001, 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051284831000000003, 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30306 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.263 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904408+0000", 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.2399999999999999e-07 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.264 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904411+0000", 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6259999999999999e-06 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904417+0000", 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.2829999999999998e-06 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904422+0000", 2024-09-14T22:55:37.265 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9019999999999998e-06 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955517+0000", 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051095347999999999 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955597+0000", 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.9699999999999999e-05 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955692+0000", 2024-09-14T22:55:37.266 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.5548000000000001e-05 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30308 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1398784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35572976700001, 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051479294000000002, 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.267 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30308 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904428+0000", 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.268 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904429+0000", 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4000000000000002e-07 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904431+0000", 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1339999999999998e-06 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904437+0000", 2024-09-14T22:55:37.269 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5100000000000004e-06 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904442+0000", 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5759999999999999e-06 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955722+0000", 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051279989999999998 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955803+0000", 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0990999999999998e-05 2024-09-14T22:55:37.270 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955907+0000", 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000104553 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30314 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1069056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35569976799999, 2024-09-14T22:55:37.271 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051646554999999997, 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30314 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:37.272 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904458+0000", 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904459+0000", 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9800000000000004e-07 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.273 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904461+0000", 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5959999999999998e-06 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904468+0000", 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0779999999999998e-06 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904480+0000", 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1955e-05 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.274 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.955935+0000", 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051454516999999998 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956003+0000", 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8720000000000006e-05 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956105+0000", 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101191 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.275 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30315 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1093632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35566861500001, 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051779090999999999, 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.276 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30315 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904489+0000", 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.277 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904490+0000", 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.4600000000000005e-07 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904492+0000", 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6529999999999998e-06 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904500+0000", 2024-09-14T22:55:37.278 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0269999999999998e-06 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904505+0000", 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1839999999999998e-06 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956133+0000", 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051628061000000003 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956205+0000", 2024-09-14T22:55:37.279 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2515000000000006e-05 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956268+0000", 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3105e-05 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30316 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1423360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35564595899999, 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051969829000000002, 2024-09-14T22:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30316 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:37.281 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904512+0000", 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.01e-07 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.282 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904515+0000", 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1789999999999998e-06 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904521+0000", 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.5969999999999997e-06 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904526+0000", 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8930000000000004e-06 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.283 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956299+0000", 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051772776 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956374+0000", 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5468000000000003e-05 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956482+0000", 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000107515 2024-09-14T22:55:37.284 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30317 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1118208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35562545799999, 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160817999999998, 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.285 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30317 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904532+0000", 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.286 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904533+0000", 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7500000000000001e-07 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904535+0000", 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.035e-06 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904541+0000", 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.3559999999999998e-06 2024-09-14T22:55:37.287 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904546+0000", 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5950000000000001e-06 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956512+0000", 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.051966153000000001 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956596+0000", 2024-09-14T22:55:37.288 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.4337000000000002e-05 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956693+0000", 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6966999999999995e-05 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30318 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1142784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35560569699999, 2024-09-14T22:55:37.289 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052358415999999998, 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30318 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:37.290 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904552+0000", 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904553+0000", 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9900000000000001e-07 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904555+0000", 2024-09-14T22:55:37.291 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3889999999999999e-06 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904562+0000", 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0559999999999999e-06 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904567+0000", 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5889999999999996e-06 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.292 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956727+0000", 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052160091999999998 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956810+0000", 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2847000000000001e-05 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956911+0000", 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000101044 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.293 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30319 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1167360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35558491200001, 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053290558000000002, 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.294 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30319 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.295 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904573+0000", 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.6800000000000001e-07 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904576+0000", 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2010000000000002e-06 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904582+0000", 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.0650000000000004e-06 2024-09-14T22:55:37.296 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904593+0000", 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1698999999999999e-05 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.956963+0000", 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.052369297000000002 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957037+0000", 2024-09-14T22:55:37.297 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.4181e-05 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957864+0000", 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00082674699999999996 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30325 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1191936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:37.298 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35555249999999, 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053454557999999999, 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30325 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:37.299 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904605+0000", 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904606+0000", 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0699999999999997e-07 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.300 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904608+0000", 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.5299999999999999e-06 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904615+0000", 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7279999999999998e-06 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904620+0000", 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9219999999999996e-06 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.301 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957893+0000", 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053273187999999999 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.957964+0000", 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0425000000000004e-05 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958060+0000", 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.6257999999999999e-05 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.302 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30326 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1241088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.355531101, 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053642131000000003, 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30326 2024-09-14T22:55:37.303 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.304 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904627+0000", 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.7e-07 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904633+0000", 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.1589999999999998e-06 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904641+0000", 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3610000000000003e-06 2024-09-14T22:55:37.305 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904646+0000", 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.8860000000000003e-06 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958091+0000", 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053445380000000001 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958174+0000", 2024-09-14T22:55:37.306 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3159999999999997e-05 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958269+0000", 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4815000000000006e-05 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30327 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1216512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35546970199999, 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053787468999999997, 2024-09-14T22:55:37.307 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30327 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.308 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904688+0000", 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904689+0000", 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.61e-07 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904700+0000", 2024-09-14T22:55:37.309 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1228e-05 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904711+0000", 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.0762e-05 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904718+0000", 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8290000000000002e-06 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.310 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958299+0000", 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053581773999999999 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958377+0000", 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7323999999999994e-05 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958476+0000", 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.8890999999999999e-05 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.311 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30328 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1265664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.355429789, 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053952348999999997, 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.312 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30328 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904728+0000", 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904729+0000", 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5499999999999998e-07 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904731+0000", 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904739+0000", 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5340000000000002e-06 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.314 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904750+0000", 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.1555e-05 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958504+0000", 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053753512000000003 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958587+0000", 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.2795999999999998e-05 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.315 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958680+0000", 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.3890999999999999e-05 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30329 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1290240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.355400033, 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054116275999999998, 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.316 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30329 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.317 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904758+0000", 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4499999999999997e-07 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904761+0000", 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.2510000000000001e-06 2024-09-14T22:55:37.318 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904768+0000", 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.2110000000000001e-06 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904773+0000", 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.1209999999999998e-06 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958710+0000", 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.053937300000000001 2024-09-14T22:55:37.319 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958791+0000", 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0756999999999999e-05 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958874+0000", 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.3190999999999997e-05 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.320 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30335 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1314816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.355369798, 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054262619999999998, 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30335 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.321 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904788+0000", 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904789+0000", 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0200000000000002e-07 2024-09-14T22:55:37.322 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904791+0000", 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.0820000000000001e-06 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904800+0000", 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4870000000000008e-06 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904805+0000", 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.0429999999999998e-06 2024-09-14T22:55:37.323 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958907+0000", 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054101665 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.958981+0000", 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.3849000000000004e-05 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959051+0000", 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9992000000000003e-05 2024-09-14T22:55:37.324 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30336 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1339392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.355345059, 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054426343000000002, 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.325 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30336 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.326 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904813+0000", 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.4000000000000002e-07 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904816+0000", 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3130000000000001e-06 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904824+0000", 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0749999999999998e-06 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.327 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904829+0000", 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3489999999999999e-06 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959087+0000", 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054257900999999997 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959162+0000", 2024-09-14T22:55:37.328 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.5036999999999993e-05 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959239+0000", 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.7228e-05 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30337 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1363968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.355321629, 2024-09-14T22:55:37.329 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054693674999999997, 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30337 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.330 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904836+0000", 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904837+0000", 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.5900000000000002e-07 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.331 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904839+0000", 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3379999999999999e-06 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904846+0000", 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1960000000000002e-06 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904852+0000", 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.3199999999999999e-06 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.332 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959365+0000", 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054513683 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959435+0000", 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9790999999999996e-05 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959530+0000", 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.4888000000000004e-05 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.333 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30338 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1388544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.35522589300001, 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054836817000000003, 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.334 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30338 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904932+0000", 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.335 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904933+0000", 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.9400000000000005e-07 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904936+0000", 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.2090000000000001e-06 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904945+0000", 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.9879999999999993e-06 2024-09-14T22:55:37.336 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904951+0000", 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.4790000000000001e-06 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959572+0000", 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054620323999999998 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959652+0000", 2024-09-14T22:55:37.337 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.0500000000000005e-05 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959769+0000", 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000116723 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30339 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1413120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.355196971, 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055059214000000002, 2024-09-14T22:55:37.338 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30339 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.339 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904961+0000", 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.6600000000000002e-07 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904964+0000", 2024-09-14T22:55:37.340 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3819999999999998e-06 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904971+0000", 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.1690000000000003e-06 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904976+0000", 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 4.9520000000000002e-06 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959820+0000", 2024-09-14T22:55:37.341 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.054844340999999998 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.959923+0000", 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.00010246700000000001 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960020+0000", 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 9.7436999999999996e-05 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.342 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:30340 3.8 3:1203a3d8:::rbd_data.11354ec77855.0000000000000006:6 [read 1437696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 135.355173646, 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055238700000000002, 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "started", 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 30340 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.343 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904984+0000", 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904985+0000", 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 3.9400000000000001e-07 2024-09-14T22:55:37.344 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904987+0000", 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.1799999999999999e-06 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904994+0000", 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 7.0899999999999999e-06 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.904999+0000", 2024-09-14T22:55:37.345 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 5.2020000000000003e-06 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960048+0000", 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.055048875999999997 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "started", 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960117+0000", 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.8849000000000004e-05 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.346 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:21.960223+0000", 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.000106109 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "description": "osd_op(client.4405.0:135241 3.8 3.2ca5bf08 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "initiated_at": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "age": 112.222475903, 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87103006199999999, 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "continuous": false, 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "type_data": { 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "flag_point": "reached pg", 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "client_info": { 2024-09-14T22:55:37.347 INFO:teuthology.orchestra.run.smithi076.stdout: "client": "client.4405", 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "tid": 135241 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "events": [ 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "initiated", 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "header_read", 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037682+0000", 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0 2024-09-14T22:55:37.348 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "throttled", 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037684+0000", 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 2.3089999999999998e-06 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "all_read", 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037694+0000", 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.006e-05 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "dispatched", 2024-09-14T22:55:37.349 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037713+0000", 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.8204000000000001e-05 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037721+0000", 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 8.6759999999999996e-06 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037790+0000", 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.9041999999999994e-05 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "waiting for peered", 2024-09-14T22:55:37.350 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.037797+0000", 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 6.7220000000000002e-06 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "reached_pg", 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908694+0000", 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 0.87089723100000005 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "event": "done", 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "time": "2024-09-14T22:53:45.908712+0000", 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: "duration": 1.7818000000000001e-05 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.351 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-14T22:55:37.352 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-14T22:55:37.352 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:55:37.353 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T22:55:37.470 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:37.470 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:37.470 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:37.470 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:37.479 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T22:55:37.582 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:37.582 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:37.582 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:37.582 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:37.582 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:37.591 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T22:55:37.695 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:37.695 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:37.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:37.695 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:110291 3.2 3:454eb293:::rbd_data.113528220df0.000000000000017f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 125.797010348, 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85203433200000001, 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 110291 2024-09-14T22:55:37.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895951+0000", 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895953+0000", 2024-09-14T22:55:37.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.423e-06 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895959+0000", 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0689999999999998e-06 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895971+0000", 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1853e-05 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.895978+0000", 2024-09-14T22:55:37.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5619999999999998e-06 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896040+0000", 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2067000000000001e-05 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:31.896046+0000", 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1299999999999998e-06 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746691+0000", 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.85064467099999996 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746839+0000", 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014787099999999999 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 6", 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.746981+0000", 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000142328 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747391+0000", 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00040980600000000003 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747874+0000", 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00048334300000000001 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747910+0000", 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.578e-05 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:32.747986+0000", 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5428999999999999e-05 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127351 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4128768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.171216229, 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053330456999999998, 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.702 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127351 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521745+0000", 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521802+0000", 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6821999999999999e-05 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521806+0000", 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0199999999999996e-06 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521815+0000", 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4500000000000004e-06 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521821+0000", 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9800000000000003e-06 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574890+0000", 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053068843999999997 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.574978+0000", 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8222999999999999e-05 2024-09-14T22:55:37.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575076+0000", 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127353 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4136960~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.171128237, 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053443138000000001, 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.706 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127353 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.521833+0000", 2024-09-14T22:55:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522031+0000", 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000197918 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522036+0000", 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522045+0000", 2024-09-14T22:55:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8510000000000005e-06 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522052+0000", 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9e-06 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575107+0000", 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053055230000000002 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575186+0000", 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8926999999999998e-05 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575277+0000", 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0678000000000002e-05 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127355 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4145152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.17089432, 2024-09-14T22:55:37.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053390838000000003, 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127355 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:37.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522067+0000", 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522217+0000", 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014989199999999999 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522221+0000", 2024-09-14T22:55:37.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7799999999999998e-06 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522229+0000", 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4239999999999993e-06 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522237+0000", 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4139999999999997e-06 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575313+0000", 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053075991000000003 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575378+0000", 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5284999999999996e-05 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575458+0000", 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0051999999999999e-05 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127357 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4153344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.170713425, 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053415808000000002, 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127357 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522248+0000", 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522517+0000", 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00026881099999999998 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522521+0000", 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.42e-06 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522531+0000", 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6549999999999999e-06 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522536+0000", 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8629999999999999e-06 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575497+0000", 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052960874999999998 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575574+0000", 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7473999999999997e-05 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575664+0000", 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9710000000000007e-05 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.718 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127359 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4161536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.17041392199999, 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053355829, 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.719 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127359 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522548+0000", 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522818+0000", 2024-09-14T22:55:37.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270413 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522823+0000", 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6890000000000003e-06 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522835+0000", 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1671999999999999e-05 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522842+0000", 2024-09-14T22:55:37.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3810000000000001e-06 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575704+0000", 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052862129000000001 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575793+0000", 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9021e-05 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.575904+0000", 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110524 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127363 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4177920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.17006771200001, 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053591417000000002, 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.723 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127363 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.522894+0000", 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523222+0000", 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032850000000000002 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523227+0000", 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6480000000000002e-06 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523237+0000", 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0247e-05 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523244+0000", 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.81e-06 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576373+0000", 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053129296999999999 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576428+0000", 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4486e-05 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576485+0000", 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7429000000000002e-05 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127365 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4186112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.16970398399999, 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053393901000000001, 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.727 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127365 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523258+0000", 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523759+0000", 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050171500000000002 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523764+0000", 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6460000000000001e-06 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523771+0000", 2024-09-14T22:55:37.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1840000000000002e-06 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523781+0000", 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576503+0000", 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052722189000000003 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576550+0000", 2024-09-14T22:55:37.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6836999999999999e-05 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.576652+0000", 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010145999999999999 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127378 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3952640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.16911436700001, 2024-09-14T22:55:37.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053402812000000001, 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127378 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.523847+0000", 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524017+0000", 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000169264 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524020+0000", 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8489999999999999e-06 2024-09-14T22:55:37.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524032+0000", 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1718e-05 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524039+0000", 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5320000000000003e-06 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577063+0000", 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053023831 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577145+0000", 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2018e-05 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577250+0000", 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010560000000000001 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.735 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127380 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3960832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.16891213, 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053414370000000003, 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127380 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524050+0000", 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524213+0000", 2024-09-14T22:55:37.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00016305599999999999 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524216+0000", 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5990000000000002e-06 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524224+0000", 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3119999999999996e-06 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524229+0000", 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7740000000000001e-06 2024-09-14T22:55:37.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577287+0000", 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053057752 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577357+0000", 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0012999999999995e-05 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577464+0000", 2024-09-14T22:55:37.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000106864 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127382 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.168722319, 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053441339999999997, 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.740 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127382 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524239+0000", 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524517+0000", 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027798500000000002 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524521+0000", 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9729999999999999e-06 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524529+0000", 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1709999999999996e-06 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524540+0000", 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1467999999999999e-05 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577499+0000", 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052958768000000003 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577570+0000", 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1236999999999996e-05 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577681+0000", 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110738 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127384 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3977216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.16841031600001, 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053334828000000001, 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.744 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127384 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524551+0000", 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524758+0000", 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020671700000000001 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524762+0000", 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2409999999999997e-06 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524773+0000", 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0633e-05 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524781+0000", 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3359999999999995e-06 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577713+0000", 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052931671999999999 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577783+0000", 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0020999999999999e-05 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577886+0000", 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010320800000000001 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127386 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3985408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.168167829, 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053301841000000003, 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.748 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127386 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524794+0000", 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524920+0000", 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012606600000000001 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524923+0000", 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4460000000000001e-06 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524935+0000", 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1800000000000001e-05 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524942+0000", 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2060000000000001e-06 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577924+0000", 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052981523000000003 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.577992+0000", 2024-09-14T22:55:37.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7612000000000003e-05 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578096+0000", 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000104188 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127388 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.168008867, 2024-09-14T22:55:37.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053343249000000002, 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127388 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.524953+0000", 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525326+0000", 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037322000000000001 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525330+0000", 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.092e-06 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525340+0000", 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4560000000000006e-06 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525346+0000", 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7739999999999999e-06 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578128+0000", 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052781806000000001 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578194+0000", 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5462000000000004e-05 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578296+0000", 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102439 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127394 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:37.756 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.16737459700001, 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053308742999999999, 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127394 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:37.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525587+0000", 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525767+0000", 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018025800000000001 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525771+0000", 2024-09-14T22:55:37.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.065e-06 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525779+0000", 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4839999999999998e-06 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525785+0000", 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5610000000000004e-06 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578729+0000", 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052943151000000001 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578793+0000", 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4105999999999999e-05 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578896+0000", 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103118 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.760 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127396 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4026368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.167160487, 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053289442999999999, 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127396 2024-09-14T22:55:37.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525801+0000", 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525910+0000", 2024-09-14T22:55:37.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108799 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525918+0000", 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3380000000000005e-06 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525925+0000", 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1400000000000002e-06 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525930+0000", 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0929999999999996e-06 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578933+0000", 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053003296999999998 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.578988+0000", 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4746000000000001e-05 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579091+0000", 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010302999999999999 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127398 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4034560~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.167024017, 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053342485000000002, 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.765 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127398 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.525938+0000", 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526158+0000", 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022074 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526163+0000", 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3309999999999997e-06 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526171+0000", 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6489999999999997e-06 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.767 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526178+0000", 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.72e-06 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579123+0000", 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052944415000000002 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579178+0000", 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5377000000000002e-05 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579280+0000", 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102253 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:127400 3.16 3:6c1f5010:::rbd_data.1135aa5cee52.0000000000000004:c [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 120.166771537, 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.053273907000000002, 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.769 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 127400 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526190+0000", 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526639+0000", 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044912400000000002 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526643+0000", 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0520000000000003e-06 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526652+0000", 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1880000000000002e-06 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.526658+0000", 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3899999999999998e-06 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579312+0000", 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.052653815999999999 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579362+0000", 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0417000000000002e-05 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:37.579464+0000", 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010192 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:194921 3.5 3:a5a6025d:::rbd_data.113513bad187.000000000000000f:head [set-alloc-hint object_size 16777216 write_size 16777216,write 1046432~2144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-14T22:55:37.773 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 70.067873212999999, 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.55453338900000004, 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 194921 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625088+0000", 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625091+0000", 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0250000000000001e-06 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625100+0000", 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0850000000000003e-06 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625116+0000", 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6543000000000001e-05 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625126+0000", 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6059999999999992e-06 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625198+0000", 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2655000000000001e-05 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:27.625209+0000", 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0845999999999999e-05 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177555+0000", 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.552345643 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177706+0000", 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015064500000000001 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 3", 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.177864+0000", 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015862600000000001 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.178719+0000", 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085470000000000001 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179555+0000", 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083626499999999999 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179594+0000", 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8909999999999998e-05 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:28.179622+0000", 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7840000000000001e-05 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204475 3.16 3:69f3e259:::rbd_data.113547a9932.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2990301~43693] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.572386584, 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.062079224000000002, 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204475 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.780 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120575+0000", 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.120577+0000", 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7269999999999999e-06 2024-09-14T22:55:37.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144351+0000", 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.023774170000000001 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144370+0000", 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8821000000000002e-05 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144381+0000", 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1379999999999999e-05 2024-09-14T22:55:37.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144452+0000", 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0538999999999997e-05 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144577+0000", 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0001248 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 7", 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.144971+0000", 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039406100000000002 2024-09-14T22:55:37.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.180455+0000", 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035484349999999998 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182538+0000", 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002082965 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182603+0000", 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5060000000000004e-05 2024-09-14T22:55:37.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.182654+0000", 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1350999999999997e-05 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:37.785 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:37.786 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T22:55:37.889 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:37.889 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:37.889 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:37.889 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:37.898 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T22:55:38.001 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:38.001 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:38.001 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:38.001 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:38.001 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:38.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T22:55:38.081 INFO:teuthology.orchestra.run.smithi076.stderr:set 2.5 pg_upmap mapping to [7,6] 2024-09-14T22:55:38.115 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35863 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.03199854600001, 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096033999999997, 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.116 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35863 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081742+0000", 2024-09-14T22:55:38.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081928+0000", 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018675500000000001 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081932+0000", 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4010000000000001e-06 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081944+0000", 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1914e-05 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081950+0000", 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1349999999999997e-06 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130597+0000", 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048646656000000003 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130708+0000", 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111774 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130838+0000", 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012939900000000001 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.120 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35864 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 1669120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.03178065, 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049100159999999997, 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35864 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.081960+0000", 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083919+0000", 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019591970000000002 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083923+0000", 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6240000000000001e-06 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083934+0000", 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0315e-05 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.083940+0000", 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2380000000000002e-06 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130885+0000", 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046944789000000001 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.130965+0000", 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0024000000000005e-05 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.131060+0000", 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4973e-05 2024-09-14T22:55:38.124 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35914 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1726464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02909700399999, 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049012435, 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.125 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35914 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084643+0000", 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084644+0000", 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9800000000000003e-07 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084647+0000", 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0120000000000001e-06 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084654+0000", 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7379999999999997e-06 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084659+0000", 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3890000000000004e-06 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133482+0000", 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048823363000000002 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133556+0000", 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3703999999999998e-05 2024-09-14T22:55:38.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133656+0000", 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9531000000000006e-05 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35915 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1751040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02893858900001, 2024-09-14T22:55:38.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049078507, 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35915 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084802+0000", 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8999999999999996e-07 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084805+0000", 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9399999999999998e-06 2024-09-14T22:55:38.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084814+0000", 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4549999999999995e-06 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084818+0000", 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6360000000000002e-06 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133697+0000", 2024-09-14T22:55:38.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048878562 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133779+0000", 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2261000000000004e-05 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133880+0000", 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101063 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35916 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1775616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02891132299999, 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049194603000000003, 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35916 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.134 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084829+0000", 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.96e-07 2024-09-14T22:55:38.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084831+0000", 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.787e-06 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084835+0000", 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6370000000000001e-06 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.084838+0000", 2024-09-14T22:55:38.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9500000000000001e-06 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133909+0000", 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049071851999999999 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.133963+0000", 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3297000000000001e-05 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134024+0000", 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0783999999999999e-05 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35917 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02870418200001, 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049108893000000001, 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35917 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085036+0000", 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085037+0000", 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7400000000000002e-07 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085041+0000", 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6559999999999998e-06 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085049+0000", 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1850000000000004e-06 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085055+0000", 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9379999999999997e-06 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134048+0000", 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048993162 2024-09-14T22:55:38.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134089+0000", 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0837000000000002e-05 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134145+0000", 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6341000000000001e-05 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.143 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35918 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1800192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.143 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:38.143 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02866991799999, 2024-09-14T22:55:38.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049195715000000001, 2024-09-14T22:55:38.143 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.143 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35918 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085070+0000", 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085071+0000", 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5899999999999996e-07 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085073+0000", 2024-09-14T22:55:38.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5519999999999999e-06 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085079+0000", 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8989999999999998e-06 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.085084+0000", 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134172+0000", 2024-09-14T22:55:38.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.04908767 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134211+0000", 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8733000000000003e-05 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.134266+0000", 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.537e-05 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35943 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1873920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02703787199999, 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049256877999999997, 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35943 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086702+0000", 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086703+0000", 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9500000000000002e-07 2024-09-14T22:55:38.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086706+0000", 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5050000000000002e-06 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086713+0000", 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2439999999999996e-06 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086718+0000", 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.361e-06 2024-09-14T22:55:38.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135530+0000", 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048812107 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135620+0000", 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9284999999999996e-05 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.135959+0000", 2024-09-14T22:55:38.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033978099999999999 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35944 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1923072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02701325999999, 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049399302999999999, 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.152 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35944 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:38.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086727+0000", 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9000000000000002e-07 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086730+0000", 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1330000000000002e-06 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086736+0000", 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1759999999999998e-06 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086742+0000", 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2519999999999996e-06 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136002+0000", 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049260106999999997 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136052+0000", 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0272000000000003e-05 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136126+0000", 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3973000000000005e-05 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35946 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.156 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.026878974, 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049403715000000001, 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35946 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086861+0000", 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086862+0000", 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7599999999999997e-07 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086865+0000", 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2729999999999998e-06 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086873+0000", 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9109999999999993e-06 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.086879+0000", 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5790000000000003e-06 2024-09-14T22:55:38.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136154+0000", 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049275332999999998 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136199+0000", 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4511999999999997e-05 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136265+0000", 2024-09-14T22:55:38.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6431000000000001e-05 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35966 3.10 3:0fede8e9:::rbd_data.11354ec77855.0000000000000007:6 [read 2078720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02572708100001, 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048970337000000003, 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.161 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35966 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088013+0000", 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088014+0000", 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8400000000000004e-07 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088017+0000", 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8320000000000002e-06 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088022+0000", 2024-09-14T22:55:38.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0490000000000003e-06 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088026+0000", 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7500000000000001e-06 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136828+0000", 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048802254000000003 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136888+0000", 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0279999999999999e-05 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.136984+0000", 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5488000000000005e-05 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35967 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1972224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:38.165 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.025703177, 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049091315000000003, 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35967 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:38.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088037+0000", 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088038+0000", 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4900000000000001e-07 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088040+0000", 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.334e-06 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088052+0000", 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2288e-05 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088061+0000", 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6279999999999994e-06 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137014+0000", 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048952939000000001 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137072+0000", 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8041999999999998e-05 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137128+0000", 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6635000000000002e-05 2024-09-14T22:55:38.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35968 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1996800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02566965700001, 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049333103000000003, 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.170 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35968 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088071+0000", 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3799999999999998e-07 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088073+0000", 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9300000000000002e-06 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088077+0000", 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8870000000000002e-06 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088080+0000", 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8600000000000001e-06 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137152+0000", 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049072751999999997 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137279+0000", 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012624899999999999 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137404+0000", 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012508699999999999 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.174 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35969 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2021376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02551501100001, 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049420028999999997, 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35969 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088225+0000", 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088227+0000", 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3030000000000001e-06 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088231+0000", 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2379999999999999e-06 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088242+0000", 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1351e-05 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088249+0000", 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0469999999999996e-06 2024-09-14T22:55:38.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137433+0000", 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049184235999999999 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137536+0000", 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000102946 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137645+0000", 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010890799999999999 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.178 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35970 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2045952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02547383699999, 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049612482999999999, 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35970 2024-09-14T22:55:38.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088266+0000", 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088267+0000", 2024-09-14T22:55:38.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5400000000000001e-07 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088270+0000", 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6060000000000001e-06 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088277+0000", 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4370000000000001e-06 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088286+0000", 2024-09-14T22:55:38.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137698+0000", 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049411297999999999 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137780+0000", 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2226999999999997e-05 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137879+0000", 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9098000000000006e-05 2024-09-14T22:55:38.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35971 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.02544567699999, 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049831568999999999, 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.183 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35971 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088295+0000", 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999999e-07 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088297+0000", 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2800000000000002e-06 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088310+0000", 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2897e-05 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088317+0000", 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3539999999999996e-06 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137930+0000", 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049613530000000003 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.137981+0000", 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0875000000000003e-05 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138126+0000", 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014518500000000001 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:35972 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 2095104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 136.025256286, 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049816862000000003, 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.187 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 35972 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088484+0000", 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088485+0000", 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088488+0000", 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.151e-06 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088497+0000", 2024-09-14T22:55:38.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5949999999999999e-06 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.088502+0000", 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6570000000000002e-06 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138169+0000", 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049667013000000003 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138236+0000", 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6279999999999996e-05 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:22.138301+0000", 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5357000000000005e-05 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64057 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 374784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.191 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 134.10916645899999, 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049093995000000001, 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64057 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004574+0000", 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.72e-07 2024-09-14T22:55:38.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004576+0000", 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0080000000000001e-06 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004583+0000", 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.601e-06 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004587+0000", 2024-09-14T22:55:38.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5379999999999996e-06 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053469+0000", 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.048881282999999998 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053556+0000", 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7016999999999997e-05 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053668+0000", 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000112176 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:64096 3.10 3:08414bcd:::rbd_data.11354ec77855.0000000000000008:6 [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 134.10914662799999, 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049315142999999999, 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.196 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 64096 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004594+0000", 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.77e-07 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004596+0000", 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.165e-06 2024-09-14T22:55:38.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004603+0000", 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.596e-06 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.004607+0000", 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6040000000000003e-06 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053704+0000", 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049096956999999997 2024-09-14T22:55:38.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053799+0000", 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4182000000000001e-05 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:24.053909+0000", 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110262 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.200 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:204731 3.10 3:0ff1e66c:::rbd_data.113547a9932.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3909880~23170] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.909440302999997, 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.056235467999999997, 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 204731 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204300+0000", 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.204301+0000", 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.167e-06 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220068+0000", 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015767275000000001 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220094+0000", 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5259e-05 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220121+0000", 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7200999999999999e-05 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220188+0000", 2024-09-14T22:55:38.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7581000000000002e-05 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220308+0000", 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119706 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.220803+0000", 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00049498000000000001 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.256568+0000", 2024-09-14T22:55:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035765039999999998 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260451+0000", 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038824720000000001 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260490+0000", 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9759999999999999e-05 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:40.260535+0000", 2024-09-14T22:55:38.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5027000000000001e-05 2024-09-14T22:55:38.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.206 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.206 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.206 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.206 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:38.207 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T22:55:38.312 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:38.312 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:38.312 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:38.312 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:38.321 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T22:55:38.424 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:38.424 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:38.424 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-14T22:55:38.424 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-14T22:55:38.424 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:38.432 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4195.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 166.0269524, 2024-09-14T22:55:38.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.15156826700000001, 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4195", 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.114:0/3378365495", 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507776+0000", 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507777+0000", 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3969999999999999e-06 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507784+0000", 2024-09-14T22:55:38.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1790000000000002e-06 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507794+0000", 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9939999999999995e-06 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507803+0000", 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6880000000000005e-06 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507868+0000", 2024-09-14T22:55:38.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4867000000000001e-05 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.507877+0000", 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7200000000000001e-06 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.653980+0000", 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.146102493 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654178+0000", 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019801700000000001 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 0", 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.654392+0000", 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000213637 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.655005+0000", 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00061323699999999998 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:38.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659271+0000", 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042657670000000002 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659313+0000", 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2406e-05 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:52:52.659344+0000", 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0864999999999997e-05 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118331 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3051520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.16229454499999, 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050212646, 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118331 2024-09-14T22:55:38.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372433+0000", 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372434+0000", 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6700000000000003e-07 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372437+0000", 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5339999999999998e-06 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372444+0000", 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5059999999999998e-06 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372450+0000", 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.49e-06 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422456+0000", 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050006631000000003 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422545+0000", 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8900999999999995e-05 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422646+0000", 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000101017 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118333 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3059712~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.162188609, 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050344195000000001, 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.548 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118333 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372539+0000", 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372540+0000", 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4900000000000005e-07 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372544+0000", 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4599999999999999e-06 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372555+0000", 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1096e-05 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372562+0000", 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8029999999999999e-06 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422693+0000", 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050131357000000001 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422781+0000", 2024-09-14T22:55:38.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8603000000000005e-05 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422884+0000", 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010212700000000001 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118335 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3067904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.16211390799999, 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050516223999999998, 2024-09-14T22:55:38.552 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118335 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372614+0000", 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372615+0000", 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372618+0000", 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4979999999999998e-06 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372634+0000", 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5862000000000002e-05 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372641+0000", 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9840000000000004e-06 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.422937+0000", 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050295922999999999 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423038+0000", 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010121 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423130+0000", 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2015000000000006e-05 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118337 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3076096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:38.556 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.162028427, 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050650384999999999, 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118337 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372700+0000", 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.23e-07 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372704+0000", 2024-09-14T22:55:38.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6059999999999999e-06 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372713+0000", 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0799999999999995e-06 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372720+0000", 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4399999999999999e-06 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423162+0000", 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050441609999999998 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423258+0000", 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5982000000000004e-05 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423350+0000", 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1943999999999999e-05 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118339 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3084288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.161882041, 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050724429000000001, 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118339 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372846+0000", 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372847+0000", 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9899999999999999e-07 2024-09-14T22:55:38.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372851+0000", 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.811e-06 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372861+0000", 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372867+0000", 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8159999999999996e-06 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423392+0000", 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050524633999999999 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423477+0000", 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5427999999999996e-05 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423570+0000", 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2977000000000001e-05 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118342 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3092480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.161832041, 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050899629000000002, 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.565 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118342 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372896+0000", 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372897+0000", 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6899999999999997e-07 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372900+0000", 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1480000000000002e-06 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372909+0000", 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6460000000000005e-06 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.372921+0000", 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1712999999999999e-05 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423616+0000", 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050694544000000001 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423700+0000", 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4691999999999995e-05 2024-09-14T22:55:38.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423796+0000", 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5217000000000007e-05 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118344 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3100672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.161668877, 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050894753000000001, 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.569 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118344 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373059+0000", 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373060+0000", 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.09e-07 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373064+0000", 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9269999999999998e-06 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373074+0000", 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373081+0000", 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3939999999999998e-06 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423828+0000", 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050746832999999998 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423889+0000", 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1162999999999997e-05 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423954+0000", 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4943999999999993e-05 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118346 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3108864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:38.573 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.161630567, 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051041956999999999, 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118346 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373097+0000", 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373098+0000", 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4099999999999999e-07 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373101+0000", 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7860000000000001e-06 2024-09-14T22:55:38.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373109+0000", 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8099999999999998e-06 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373114+0000", 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.463e-06 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.423982+0000", 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050867680999999998 2024-09-14T22:55:38.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424038+0000", 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6209999999999999e-05 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424139+0000", 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010146599999999999 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.577 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118348 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3117056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.161496898, 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051078985, 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118348 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373231+0000", 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373232+0000", 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5700000000000001e-07 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373236+0000", 2024-09-14T22:55:38.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4410000000000003e-06 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373245+0000", 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9279999999999999e-06 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373267+0000", 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1362e-05 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424170+0000", 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050903060999999999 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424260+0000", 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0163999999999999e-05 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424310+0000", 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0172e-05 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118350 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3125248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.161416331, 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051127453000000003, 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118350 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373312+0000", 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8999999999999995e-07 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373316+0000", 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6090000000000001e-06 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373326+0000", 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0047999999999999e-05 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373336+0000", 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8339999999999994e-06 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424341+0000", 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051004925 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424388+0000", 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7119000000000001e-05 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424439+0000", 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1128000000000002e-05 2024-09-14T22:55:38.585 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118352 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.161319394, 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051250981000000001, 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.586 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118352 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373409+0000", 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3200000000000004e-07 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373419+0000", 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0020999999999999e-05 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373430+0000", 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.041e-05 2024-09-14T22:55:38.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373436+0000", 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.489e-06 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424466+0000", 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051029606999999998 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424552+0000", 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5624999999999994e-05 2024-09-14T22:55:38.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424660+0000", 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010809700000000001 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118354 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3141632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.161267839, 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051408991000000001, 2024-09-14T22:55:38.590 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118354 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373460+0000", 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373461+0000", 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5199999999999996e-07 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373464+0000", 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9909999999999998e-06 2024-09-14T22:55:38.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373471+0000", 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1999999999999997e-06 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.373476+0000", 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3730000000000001e-06 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424690+0000", 2024-09-14T22:55:38.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.051213349999999998 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424780+0000", 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0031999999999995e-05 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.424869+0000", 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9393000000000003e-05 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118413 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3743744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.15814321000001, 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050162521000000002, 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118413 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376585+0000", 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6000000000000004e-07 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376595+0000", 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6679999999999996e-06 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376603+0000", 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3119999999999994e-06 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376609+0000", 2024-09-14T22:55:38.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8819999999999998e-06 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426612+0000", 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050002369999999997 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426668+0000", 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6097999999999997e-05 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426747+0000", 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9630999999999997e-05 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118415 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3751936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.158011597, 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050238427000000002, 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.599 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118415 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376716+0000", 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376717+0000", 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6700000000000002e-07 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376722+0000", 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2200000000000003e-06 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376733+0000", 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1286e-05 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.376740+0000", 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2359999999999998e-06 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426777+0000", 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050037188000000003 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426857+0000", 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9511999999999994e-05 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426955+0000", 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8117999999999998e-05 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:118417 3.15 3:af34c540:::rbd_data.1135aa5cee52.0000000000000003:c [read 3760128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 122.157725185, 2024-09-14T22:55:38.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050107736999999999, 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 118417 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377003+0000", 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377004+0000", 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8599999999999997e-07 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377008+0000", 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9700000000000001e-06 2024-09-14T22:55:38.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377022+0000", 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4301000000000001e-05 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.377029+0000", 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1620000000000002e-06 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.426985+0000", 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.049955557999999997 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427047+0000", 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1977000000000006e-05 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:53:36.427111+0000", 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3983e-05 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153281 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 95.099688931000003, 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061113032999999997, 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153281 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435039+0000", 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435040+0000", 2024-09-14T22:55:38.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0640000000000001e-06 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435045+0000", 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9699999999999998e-06 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435056+0000", 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0806e-05 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435061+0000", 2024-09-14T22:55:38.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.434e-06 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435108+0000", 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6152000000000001e-05 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435158+0000", 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0578000000000003e-05 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:38.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435353+0000", 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019449699999999999 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436247+0000", 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00089480799999999997 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496033+0000", 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059785903000000001 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496101+0000", 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7387999999999999e-05 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496152+0000", 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1433e-05 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153282 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 95.099649103000004, 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061379632000000003, 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153282 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435079+0000", 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5100000000000001e-07 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435082+0000", 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3700000000000002e-06 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435086+0000", 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4039999999999997e-06 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435089+0000", 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0400000000000001e-06 2024-09-14T22:55:38.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435664+0000", 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00057534000000000003 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435711+0000", 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.685e-05 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435987+0000", 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027590600000000002 2024-09-14T22:55:38.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436784+0000", 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000796995 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496391+0000", 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059607040999999999 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496428+0000", 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6443000000000003e-05 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496459+0000", 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0892000000000002e-05 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:153283 3.11 3:88b841d7:::rbd_object_map.1135d0236bc5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 95.099633088000004, 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.061707238999999997, 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.619 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 153283 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435095+0000", 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.35e-07 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435097+0000", 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.844e-06 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435101+0000", 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.58e-06 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.435103+0000", 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.553e-06 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436273+0000", 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001169514 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436324+0000", 2024-09-14T22:55:38.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1205000000000002e-05 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.436524+0000", 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020001599999999999 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.437380+0000", 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00085597699999999995 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496736+0000", 2024-09-14T22:55:38.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.059355814999999999 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496771+0000", 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5456000000000001e-05 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:03.496802+0000", 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1044000000000003e-05 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4405.0:157586 3.e 3:77097f43:::rbd_data.1135d0236bc5.0000000000000035:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 94.050504732999997, 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.050095488000000001, 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "continuous": false, 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4405", 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.76:0/3606122306", 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 157586 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.625 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.484223+0000", 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485923+0000", 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016993749999999999 2024-09-14T22:55:38.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485928+0000", 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1309999999999997e-06 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485945+0000", 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.732e-05 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.485951+0000", 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.789e-06 2024-09-14T22:55:38.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532897+0000", 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.046945711000000001 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.532972+0000", 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5179000000000006e-05 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for subops from 4", 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.533182+0000", 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00021018600000000001 2024-09-14T22:55:38.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_commit_rec", 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534209+0000", 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010265669999999999 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op_commit", 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534226+0000", 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7031e-05 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534281+0000", 2024-09-14T22:55:38.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5719e-05 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-14T22:54:04.534319+0000", 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.748e-05 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-14T22:55:38.630 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:39.087 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T22:55:40.099 INFO:teuthology.orchestra.run.smithi076.stderr:noscrub is unset 2024-09-14T22:55:40.115 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T22:55:41.105 INFO:teuthology.orchestra.run.smithi076.stderr:nodeep-scrub is unset 2024-09-14T22:55:43.096 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-14T22:55:43.097 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-14T22:55:43.097 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T22:55:43.097 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T22:55:43.100 INFO:tasks.ceph.osd.4:Started 2024-09-14T22:55:43.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:config diff 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:config diff get 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:config get 2024-09-14T22:55:43.209 INFO:teuthology.orchestra.run.smithi114.stderr:config help [] 2024-09-14T22:55:43.210 INFO:teuthology.orchestra.run.smithi114.stderr:config set ... 2024-09-14T22:55:43.210 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-14T22:55:43.211 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:43.212 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-14T22:55:44.532 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:55:44.530+0000 7efddb8b4640 -1 Falling back to public interface 2024-09-14T22:55:48.130 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:55:48.129+0000 7efddb8b4640 -1 osd.4 123 log_to_monitors true 2024-09-14T22:55:48.212 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:no valid command found; 10 closest matches: 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:0 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:1 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:2 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:abort 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:assert 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs debug_inject_read_zeros 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs files list 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:bluefs stats 2024-09-14T22:55:48.312 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator dump block 2024-09-14T22:55:48.313 INFO:teuthology.orchestra.run.smithi114.stderr:bluestore allocator fragmentation block 2024-09-14T22:55:48.313 INFO:teuthology.orchestra.run.smithi114.stderr:admin_socket: invalid command 2024-09-14T22:55:48.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T22:55:48.315 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-14T22:55:49.155 INFO:tasks.ceph.osd.4.smithi114.stderr:2024-09-14T22:55:49.154+0000 7efdcf663640 -1 osd.4 168 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T22:55:53.316 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T22:55:53.431 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-14T22:55:53.431 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-14T22:55:53.431 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-14T22:55:53.431 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-14T22:55:53.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T22:55:53.561 INFO:teuthology.orchestra.run.smithi076.stdout:mon.a: {} 2024-09-14T22:55:53.561 INFO:teuthology.orchestra.run.smithi076.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:55:53.570 INFO:teuthology.orchestra.run.smithi076.stdout:mon.b: {} 2024-09-14T22:55:53.570 INFO:teuthology.orchestra.run.smithi076.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:55:53.579 INFO:teuthology.orchestra.run.smithi076.stdout:mon.c: {} 2024-09-14T22:55:53.579 INFO:teuthology.orchestra.run.smithi076.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T22:55:53.590 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T22:55:54.762 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T22:55:54.778 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T22:55:55.760 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T22:55:55.777 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T22:55:56.762 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T22:55:56.780 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T22:55:57.769 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T22:55:57.787 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T22:55:58.770 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T22:55:58.789 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T22:55:59.784 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T22:55:59.800 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T22:56:00.789 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T22:56:00.804 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T22:56:01.798 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T22:56:01.814 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T22:56:02.759 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T22:56:02.774 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T22:56:03.805 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T22:56:03.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T22:56:04.807 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T22:56:04.824 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T22:56:05.817 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T22:56:05.833 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T22:56:06.819 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T22:56:06.835 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T22:56:07.827 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T22:56:07.841 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T22:56:08.832 INFO:teuthology.orchestra.run.smithi076.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T22:56:08.847 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T22:56:09.834 INFO:teuthology.orchestra.run.smithi076.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T22:56:09.850 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T22:56:09.850 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T22:56:09.850 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T22:56:09.850 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T22:56:09.850 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T22:56:09.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:56:10.129 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:10.129 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":189,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:56:09.832649+0000","last_up_change":"2024-09-14T22:55:50.117726+0000","last_in_change":"2024-09-14T22:55:31.022383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:52:50.813186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:52:55.912547+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":2.630000114440918,"raw_score_stable":2.630000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T22:53:05.643169+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":162,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6801","nonce":2488220599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6803","nonce":2488220599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6807","nonce":2488220599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6805","nonce":2488220599}]},"public_addr":"172.21.15.76:6801/2488220599","cluster_addr":"172.21.15.76:6803/2488220599","heartbeat_back_addr":"172.21.15.76:6807/2488220599","heartbeat_front_addr":"172.21.15.76:6805/2488220599","state":["exists","up"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6809","nonce":522829438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6811","nonce":522829438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6815","nonce":522829438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6813","nonce":522829438}]},"public_addr":"172.21.15.76:6809/522829438","cluster_addr":"172.21.15.76:6811/522829438","heartbeat_back_addr":"172.21.15.76:6815/522829438","heartbeat_front_addr":"172.21.15.76:6813/522829438","state":["exists","up"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6825","nonce":3214755929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6827","nonce":3214755929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6831","nonce":3214755929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6829","nonce":3214755929}]},"public_addr":"172.21.15.76:6825/3214755929","cluster_addr":"172.21.15.76:6827/3214755929","heartbeat_back_addr":"172.21.15.76:6831/3214755929","heartbeat_front_addr":"172.21.15.76:6829/3214755929","state":["exists","up"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6817","nonce":2699236014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6819","nonce":2699236014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6823","nonce":2699236014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6821","nonce":2699236014}]},"public_addr":"172.21.15.76:6817/2699236014","cluster_addr":"172.21.15.76:6819/2699236014","heartbeat_back_addr":"172.21.15.76:6823/2699236014","heartbeat_front_addr":"172.21.15.76:6821/2699236014","state":["exists","up"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":123,"up_from":169,"up_thru":172,"down_at":124,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6825","nonce":3093265760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6827","nonce":3093265760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6831","nonce":3093265760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6829","nonce":3093265760}]},"public_addr":"172.21.15.114:6825/3093265760","cluster_addr":"172.21.15.114:6827/3093265760","heartbeat_back_addr":"172.21.15.114:6831/3093265760","heartbeat_front_addr":"172.21.15.114:6829/3093265760","state":["exists","up"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6801","nonce":3469663218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6803","nonce":3469663218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6807","nonce":3469663218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6805","nonce":3469663218}]},"public_addr":"172.21.15.114:6801/3469663218","cluster_addr":"172.21.15.114:6803/3469663218","heartbeat_back_addr":"172.21.15.114:6807/3469663218","heartbeat_front_addr":"172.21.15.114:6805/3469663218","state":["exists","up"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6817","nonce":2551471767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6819","nonce":2551471767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6823","nonce":2551471767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6821","nonce":2551471767}]},"public_addr":"172.21.15.114:6817/2551471767","cluster_addr":"172.21.15.114:6819/2551471767","heartbeat_back_addr":"172.21.15.114:6823/2551471767","heartbeat_front_addr":"172.21.15.114:6821/2551471767","state":["exists","up"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6809","nonce":941014947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6811","nonce":941014947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6815","nonce":941014947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6813","nonce":941014947}]},"public_addr":"172.21.15.114:6809/941014947","cluster_addr":"172.21.15.114:6811/941014947","heartbeat_back_addr":"172.21.15.114:6815/941014947","heartbeat_front_addr":"172.21.15.114:6813/941014947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.133153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.415439+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.880448+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.879405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:47.079602+0000","dead_epoch":124},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.493580+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.693553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.442044+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[7,6]},{"pgid":"2.6","osds":[5,2]},{"pgid":"3.0","osds":[6,7]},{"pgid":"3.2","osds":[4,5]},{"pgid":"3.7","osds":[7,6]},{"pgid":"3.8","osds":[3,6]}],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:56:10.142 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T22:56:10.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:56:10.420 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:10.421 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":189,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:56:09.832649+0000","last_up_change":"2024-09-14T22:55:50.117726+0000","last_in_change":"2024-09-14T22:55:31.022383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:52:50.813186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:52:55.912547+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":2.630000114440918,"raw_score_stable":2.630000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T22:53:05.643169+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":162,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6801","nonce":2488220599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6803","nonce":2488220599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6807","nonce":2488220599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6805","nonce":2488220599}]},"public_addr":"172.21.15.76:6801/2488220599","cluster_addr":"172.21.15.76:6803/2488220599","heartbeat_back_addr":"172.21.15.76:6807/2488220599","heartbeat_front_addr":"172.21.15.76:6805/2488220599","state":["exists","up"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6809","nonce":522829438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6811","nonce":522829438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6815","nonce":522829438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6813","nonce":522829438}]},"public_addr":"172.21.15.76:6809/522829438","cluster_addr":"172.21.15.76:6811/522829438","heartbeat_back_addr":"172.21.15.76:6815/522829438","heartbeat_front_addr":"172.21.15.76:6813/522829438","state":["exists","up"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6825","nonce":3214755929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6827","nonce":3214755929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6831","nonce":3214755929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6829","nonce":3214755929}]},"public_addr":"172.21.15.76:6825/3214755929","cluster_addr":"172.21.15.76:6827/3214755929","heartbeat_back_addr":"172.21.15.76:6831/3214755929","heartbeat_front_addr":"172.21.15.76:6829/3214755929","state":["exists","up"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6817","nonce":2699236014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6819","nonce":2699236014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6823","nonce":2699236014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6821","nonce":2699236014}]},"public_addr":"172.21.15.76:6817/2699236014","cluster_addr":"172.21.15.76:6819/2699236014","heartbeat_back_addr":"172.21.15.76:6823/2699236014","heartbeat_front_addr":"172.21.15.76:6821/2699236014","state":["exists","up"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":123,"up_from":169,"up_thru":172,"down_at":124,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6825","nonce":3093265760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6827","nonce":3093265760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6831","nonce":3093265760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6829","nonce":3093265760}]},"public_addr":"172.21.15.114:6825/3093265760","cluster_addr":"172.21.15.114:6827/3093265760","heartbeat_back_addr":"172.21.15.114:6831/3093265760","heartbeat_front_addr":"172.21.15.114:6829/3093265760","state":["exists","up"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6801","nonce":3469663218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6803","nonce":3469663218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6807","nonce":3469663218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6805","nonce":3469663218}]},"public_addr":"172.21.15.114:6801/3469663218","cluster_addr":"172.21.15.114:6803/3469663218","heartbeat_back_addr":"172.21.15.114:6807/3469663218","heartbeat_front_addr":"172.21.15.114:6805/3469663218","state":["exists","up"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6817","nonce":2551471767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6819","nonce":2551471767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6823","nonce":2551471767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6821","nonce":2551471767}]},"public_addr":"172.21.15.114:6817/2551471767","cluster_addr":"172.21.15.114:6819/2551471767","heartbeat_back_addr":"172.21.15.114:6823/2551471767","heartbeat_front_addr":"172.21.15.114:6821/2551471767","state":["exists","up"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6809","nonce":941014947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6811","nonce":941014947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6815","nonce":941014947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6813","nonce":941014947}]},"public_addr":"172.21.15.114:6809/941014947","cluster_addr":"172.21.15.114:6811/941014947","heartbeat_back_addr":"172.21.15.114:6815/941014947","heartbeat_front_addr":"172.21.15.114:6813/941014947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.133153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.415439+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.880448+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.879405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:47.079602+0000","dead_epoch":124},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.493580+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.693553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.442044+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[7,6]},{"pgid":"2.6","osds":[5,2]},{"pgid":"3.0","osds":[6,7]},{"pgid":"3.2","osds":[4,5]},{"pgid":"3.7","osds":[7,6]},{"pgid":"3.8","osds":[3,6]}],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:56:10.434 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T22:56:10.434 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T22:56:10.434 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T22:56:10.435 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T22:56:10.435 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T22:56:10.436 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T22:56:10.436 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T22:56:10.436 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T22:56:10.672 INFO:teuthology.orchestra.run.smithi076.stdout:60129542189 2024-09-14T22:56:10.672 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T22:56:10.673 INFO:teuthology.orchestra.run.smithi076.stdout:60129542189 2024-09-14T22:56:10.674 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T22:56:10.677 INFO:teuthology.orchestra.run.smithi076.stdout:60129542190 2024-09-14T22:56:10.678 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T22:56:10.678 INFO:teuthology.orchestra.run.smithi076.stdout:60129542191 2024-09-14T22:56:10.679 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T22:56:10.679 INFO:teuthology.orchestra.run.smithi076.stdout:60129542189 2024-09-14T22:56:10.680 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T22:56:10.681 INFO:teuthology.orchestra.run.smithi076.stdout:725849473030 2024-09-14T22:56:10.682 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T22:56:10.682 INFO:teuthology.orchestra.run.smithi076.stdout:60129542190 2024-09-14T22:56:10.682 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T22:56:10.684 INFO:teuthology.orchestra.run.smithi076.stdout:60129542189 2024-09-14T22:56:10.684 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T22:56:11.240 INFO:teuthology.orchestra.run.smithi076.stdout:60129542189 2024-09-14T22:56:11.245 INFO:teuthology.orchestra.run.smithi076.stdout:60129542189 2024-09-14T22:56:11.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542189 for osd.5 2024-09-14T22:56:11.261 DEBUG:teuthology.parallel:result is None 2024-09-14T22:56:11.263 INFO:teuthology.orchestra.run.smithi076.stdout:60129542191 2024-09-14T22:56:11.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542189 for osd.2 2024-09-14T22:56:11.265 DEBUG:teuthology.parallel:result is None 2024-09-14T22:56:11.271 INFO:teuthology.orchestra.run.smithi076.stdout:60129542190 2024-09-14T22:56:11.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542191 got 60129542191 for osd.1 2024-09-14T22:56:11.282 DEBUG:teuthology.parallel:result is None 2024-09-14T22:56:11.286 INFO:teuthology.orchestra.run.smithi076.stdout:725849473030 2024-09-14T22:56:11.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.0 2024-09-14T22:56:11.287 DEBUG:teuthology.parallel:result is None 2024-09-14T22:56:11.290 INFO:teuthology.orchestra.run.smithi076.stdout:60129542189 2024-09-14T22:56:11.294 INFO:teuthology.orchestra.run.smithi076.stdout:60129542190 2024-09-14T22:56:11.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 725849473030 got 725849473030 for osd.4 2024-09-14T22:56:11.301 DEBUG:teuthology.parallel:result is None 2024-09-14T22:56:11.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542189 for osd.3 2024-09-14T22:56:11.306 DEBUG:teuthology.parallel:result is None 2024-09-14T22:56:11.312 INFO:teuthology.orchestra.run.smithi076.stdout:60129542189 2024-09-14T22:56:11.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.6 2024-09-14T22:56:11.313 DEBUG:teuthology.parallel:result is None 2024-09-14T22:56:11.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542189 got 60129542189 for osd.7 2024-09-14T22:56:11.325 DEBUG:teuthology.parallel:result is None 2024-09-14T22:56:11.325 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T22:56:11.325 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:56:11.640 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:56:11.653 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-14T22:56:10.819047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1385,"num_object_clones":0,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":1377,"num_read":260873,"num_read_kb":1066801,"num_write":19173,"num_write_kb":705314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":141629863,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18940,"ondisk_log_size":18940,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333300,"kb_used_data":18932,"kb_used_omap":58,"kb_used_meta":314117,"kb_avail":754641420,"statfs":{"total":773094113280,"available":772752814080,"internally_reserved":0,"allocated":19386368,"data_stored":10070853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60192,"internal_metadata":321656032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000849"},"pg_stats":[{"pgid":"3.1f","version":"162'679","reported_seq":8427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836344+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836344+0000","last_peered":"2024-09-14T22:56:09.836344+0000","last_clean":"2024-09-14T22:56:09.836344+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:56:09.836344+0000","last_undegraded":"2024-09-14T22:56:09.836344+0000","last_fullsized":"2024-09-14T22:56:09.836344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:24:55.707066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001547748,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":9382,"num_read_kb":31310,"num_write":714,"num_write_kb":7692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"160'529","reported_seq":8618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313112+0000","last_change":"2024-09-14T22:55:36.055650+0000","last_active":"2024-09-14T22:56:10.313112+0000","last_peered":"2024-09-14T22:56:10.313112+0000","last_clean":"2024-09-14T22:56:10.313112+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:56:10.313112+0000","last_undegraded":"2024-09-14T22:56:10.313112+0000","last_fullsized":"2024-09-14T22:56:10.313112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:03:40.697937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012941300000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8409,"num_read_kb":20232,"num_write":463,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"170'521","reported_seq":9286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313783+0000","last_change":"2024-09-14T22:55:51.254209+0000","last_active":"2024-09-14T22:56:10.313783+0000","last_peered":"2024-09-14T22:56:10.313783+0000","last_clean":"2024-09-14T22:56:10.313783+0000","last_became_active":"2024-09-14T22:55:51.142689+0000","last_became_peered":"2024-09-14T22:55:51.142689+0000","last_unstale":"2024-09-14T22:56:10.313783+0000","last_undegraded":"2024-09-14T22:56:10.313783+0000","last_fullsized":"2024-09-14T22:56:10.313783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:02.520144+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.057378967000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9221,"num_read_kb":38644,"num_write":441,"num_write_kb":52016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"170'649","reported_seq":3618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313632+0000","last_change":"2024-09-14T22:55:51.339722+0000","last_active":"2024-09-14T22:56:10.313632+0000","last_peered":"2024-09-14T22:56:10.313632+0000","last_clean":"2024-09-14T22:56:10.313632+0000","last_became_active":"2024-09-14T22:55:51.144955+0000","last_became_peered":"2024-09-14T22:55:51.144955+0000","last_unstale":"2024-09-14T22:56:10.313632+0000","last_undegraded":"2024-09-14T22:56:10.313632+0000","last_fullsized":"2024-09-14T22:56:10.313632+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:33:11.938624+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.037609783000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":4493,"num_read_kb":12183,"num_write":718,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"170'340","reported_seq":5375,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836439+0000","last_change":"2024-09-14T22:55:51.202792+0000","last_active":"2024-09-14T22:56:09.836439+0000","last_peered":"2024-09-14T22:56:09.836439+0000","last_clean":"2024-09-14T22:56:09.836439+0000","last_became_active":"2024-09-14T22:55:51.137980+0000","last_became_peered":"2024-09-14T22:55:51.137980+0000","last_unstale":"2024-09-14T22:56:09.836439+0000","last_undegraded":"2024-09-14T22:56:09.836439+0000","last_fullsized":"2024-09-14T22:56:09.836439+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:12.182377+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034914117000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":4616,"num_read_kb":54005,"num_write":312,"num_write_kb":18747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"155'691","reported_seq":14554,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314213+0000","last_change":"2024-09-14T22:55:53.767098+0000","last_active":"2024-09-14T22:56:10.314213+0000","last_peered":"2024-09-14T22:56:10.314213+0000","last_clean":"2024-09-14T22:56:10.314213+0000","last_became_active":"2024-09-14T22:55:53.765594+0000","last_became_peered":"2024-09-14T22:55:53.765594+0000","last_unstale":"2024-09-14T22:56:10.314213+0000","last_undegraded":"2024-09-14T22:56:10.314213+0000","last_fullsized":"2024-09-14T22:56:10.314213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:56.211691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071064000000000001,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":16209,"num_read_kb":53499,"num_write":537,"num_write_kb":29099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":18306298,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"160'485","reported_seq":16175,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835713+0000","last_change":"2024-09-14T22:55:58.780115+0000","last_active":"2024-09-14T22:56:09.835713+0000","last_peered":"2024-09-14T22:56:09.835713+0000","last_clean":"2024-09-14T22:56:09.835713+0000","last_became_active":"2024-09-14T22:55:58.778624+0000","last_became_peered":"2024-09-14T22:55:58.778624+0000","last_unstale":"2024-09-14T22:56:09.835713+0000","last_undegraded":"2024-09-14T22:56:09.835713+0000","last_fullsized":"2024-09-14T22:56:09.835713+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:43.711519+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059748170000000003,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":15953,"num_read_kb":92561,"num_write":414,"num_write_kb":38470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'675","reported_seq":2449,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312849+0000","last_change":"2024-09-14T22:55:36.056150+0000","last_active":"2024-09-14T22:56:10.312849+0000","last_peered":"2024-09-14T22:56:10.312849+0000","last_clean":"2024-09-14T22:56:10.312849+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:56:10.312849+0000","last_undegraded":"2024-09-14T22:56:10.312849+0000","last_fullsized":"2024-09-14T22:56:10.312849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:58.129978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001629828,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":2959,"num_read_kb":22740,"num_write":861,"num_write_kb":10481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"161'652","reported_seq":6924,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836420+0000","last_change":"2024-09-14T22:55:56.769135+0000","last_active":"2024-09-14T22:56:09.836420+0000","last_peered":"2024-09-14T22:56:09.836420+0000","last_clean":"2024-09-14T22:56:09.836420+0000","last_became_active":"2024-09-14T22:55:56.768221+0000","last_became_peered":"2024-09-14T22:55:56.768221+0000","last_unstale":"2024-09-14T22:56:09.836420+0000","last_undegraded":"2024-09-14T22:56:09.836420+0000","last_fullsized":"2024-09-14T22:56:09.836420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:42.862372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001569408,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":7874,"num_read_kb":34799,"num_write":817,"num_write_kb":27727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"161'433","reported_seq":9778,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313725+0000","last_change":"2024-09-14T22:55:36.057943+0000","last_active":"2024-09-14T22:56:10.313725+0000","last_peered":"2024-09-14T22:56:10.313725+0000","last_clean":"2024-09-14T22:56:10.313725+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:56:10.313725+0000","last_undegraded":"2024-09-14T22:56:10.313725+0000","last_fullsized":"2024-09-14T22:56:10.313725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:01.697087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011312640000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10352,"num_read_kb":46008,"num_write":347,"num_write_kb":32677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"161'421","reported_seq":9422,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837884+0000","last_change":"2024-09-14T22:55:36.135035+0000","last_active":"2024-09-14T22:56:09.837884+0000","last_peered":"2024-09-14T22:56:09.837884+0000","last_clean":"2024-09-14T22:56:09.837884+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:56:09.837884+0000","last_undegraded":"2024-09-14T22:56:09.837884+0000","last_fullsized":"2024-09-14T22:56:09.837884+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:30.224048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086910600000000002,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10004,"num_read_kb":43389,"num_write":402,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"170'601","reported_seq":10915,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314273+0000","last_change":"2024-09-14T22:55:51.257123+0000","last_active":"2024-09-14T22:56:10.314273+0000","last_peered":"2024-09-14T22:56:10.314273+0000","last_clean":"2024-09-14T22:56:10.314273+0000","last_became_active":"2024-09-14T22:55:51.138619+0000","last_became_peered":"2024-09-14T22:55:51.138619+0000","last_unstale":"2024-09-14T22:56:10.314273+0000","last_undegraded":"2024-09-14T22:56:10.314273+0000","last_fullsized":"2024-09-14T22:56:10.314273+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:13.451285+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.074291557999999994,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11380,"num_read_kb":66402,"num_write":623,"num_write_kb":37521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313940+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:56:10.313940+0000","last_peered":"2024-09-14T22:56:10.313940+0000","last_clean":"2024-09-14T22:56:10.313940+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:56:10.313940+0000","last_undegraded":"2024-09-14T22:56:10.313940+0000","last_fullsized":"2024-09-14T22:56:10.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"162'728","reported_seq":8066,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836307+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836307+0000","last_peered":"2024-09-14T22:56:09.836307+0000","last_clean":"2024-09-14T22:56:09.836307+0000","last_became_active":"2024-09-14T22:55:32.040243+0000","last_became_peered":"2024-09-14T22:55:32.040243+0000","last_unstale":"2024-09-14T22:56:09.836307+0000","last_undegraded":"2024-09-14T22:56:09.836307+0000","last_fullsized":"2024-09-14T22:56:09.836307+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:28:24.384824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012928849999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":9421,"num_read_kb":42199,"num_write":751,"num_write_kb":24667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":310,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313779+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:56:10.313779+0000","last_peered":"2024-09-14T22:56:10.313779+0000","last_clean":"2024-09-14T22:56:10.313779+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:56:10.313779+0000","last_undegraded":"2024-09-14T22:56:10.313779+0000","last_fullsized":"2024-09-14T22:56:10.313779+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"156'335","reported_seq":2125,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837835+0000","last_change":"2024-09-14T22:55:36.134620+0000","last_active":"2024-09-14T22:56:09.837835+0000","last_peered":"2024-09-14T22:56:09.837835+0000","last_clean":"2024-09-14T22:56:09.837835+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:56:09.837835+0000","last_undegraded":"2024-09-14T22:56:09.837835+0000","last_fullsized":"2024-09-14T22:56:09.837835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:44.383091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079363000000000003,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":1234,"num_read_kb":15244,"num_write":279,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837945+0000","last_change":"2024-09-14T22:55:39.451745+0000","last_active":"2024-09-14T22:56:09.837945+0000","last_peered":"2024-09-14T22:56:09.837945+0000","last_clean":"2024-09-14T22:56:09.837945+0000","last_became_active":"2024-09-14T22:55:39.451421+0000","last_became_peered":"2024-09-14T22:55:39.451421+0000","last_unstale":"2024-09-14T22:56:09.837945+0000","last_undegraded":"2024-09-14T22:56:09.837945+0000","last_fullsized":"2024-09-14T22:56:09.837945+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:48.524842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"161'844","reported_seq":14297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835861+0000","last_change":"2024-09-14T22:55:58.778184+0000","last_active":"2024-09-14T22:56:09.835861+0000","last_peered":"2024-09-14T22:56:09.835861+0000","last_clean":"2024-09-14T22:56:09.835861+0000","last_became_active":"2024-09-14T22:55:58.776278+0000","last_became_peered":"2024-09-14T22:55:58.776278+0000","last_unstale":"2024-09-14T22:56:09.835861+0000","last_undegraded":"2024-09-14T22:56:09.835861+0000","last_fullsized":"2024-09-14T22:56:09.835861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:19:29.875793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992360000000002,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":15582,"num_read_kb":40098,"num_write":1081,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":15786495,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835949+0000","last_change":"2024-09-14T22:55:58.776581+0000","last_active":"2024-09-14T22:56:09.835949+0000","last_peered":"2024-09-14T22:56:09.835949+0000","last_clean":"2024-09-14T22:56:09.835949+0000","last_became_active":"2024-09-14T22:55:58.776284+0000","last_became_peered":"2024-09-14T22:55:58.776284+0000","last_unstale":"2024-09-14T22:56:09.835949+0000","last_undegraded":"2024-09-14T22:56:09.835949+0000","last_fullsized":"2024-09-14T22:56:09.835949+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:35.328965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"161'434","reported_seq":15790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313853+0000","last_change":"2024-09-14T22:55:36.057245+0000","last_active":"2024-09-14T22:56:10.313853+0000","last_peered":"2024-09-14T22:56:10.313853+0000","last_clean":"2024-09-14T22:56:10.313853+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:56:10.313853+0000","last_undegraded":"2024-09-14T22:56:10.313853+0000","last_fullsized":"2024-09-14T22:56:10.313853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:39:45.146222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055810799999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14507,"num_read_kb":48865,"num_write":357,"num_write_kb":19125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"162'1001","reported_seq":6842,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312863+0000","last_change":"2024-09-14T22:55:36.109343+0000","last_active":"2024-09-14T22:56:10.312863+0000","last_peered":"2024-09-14T22:56:10.312863+0000","last_clean":"2024-09-14T22:56:10.312863+0000","last_became_active":"2024-09-14T22:54:47.747387+0000","last_became_peered":"2024-09-14T22:54:47.747387+0000","last_unstale":"2024-09-14T22:56:10.312863+0000","last_undegraded":"2024-09-14T22:56:10.312863+0000","last_fullsized":"2024-09-14T22:56:10.312863+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:27.684254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356700000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":9071,"num_read_kb":57942,"num_write":1041,"num_write_kb":41152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":29933802,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313688+0000","last_change":"2024-09-14T22:55:32.043590+0000","last_active":"2024-09-14T22:56:10.313688+0000","last_peered":"2024-09-14T22:56:10.313688+0000","last_clean":"2024-09-14T22:56:10.313688+0000","last_became_active":"2024-09-14T22:55:32.040648+0000","last_became_peered":"2024-09-14T22:55:32.040648+0000","last_unstale":"2024-09-14T22:56:10.313688+0000","last_undegraded":"2024-09-14T22:56:10.313688+0000","last_fullsized":"2024-09-14T22:56:10.313688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:45:24.150360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"161'370","reported_seq":1908,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313752+0000","last_change":"2024-09-14T22:55:36.062326+0000","last_active":"2024-09-14T22:56:10.313752+0000","last_peered":"2024-09-14T22:56:10.313752+0000","last_clean":"2024-09-14T22:56:10.313752+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:56:10.313752+0000","last_undegraded":"2024-09-14T22:56:10.313752+0000","last_fullsized":"2024-09-14T22:56:10.313752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:43:31.326081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140828,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":1390,"num_read_kb":9489,"num_write":336,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":363,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312875+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:56:10.312875+0000","last_peered":"2024-09-14T22:56:10.312875+0000","last_clean":"2024-09-14T22:56:10.312875+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:56:10.312875+0000","last_undegraded":"2024-09-14T22:56:10.312875+0000","last_fullsized":"2024-09-14T22:56:10.312875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"170'632","reported_seq":9392,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836259+0000","last_change":"2024-09-14T22:55:51.310755+0000","last_active":"2024-09-14T22:56:09.836259+0000","last_peered":"2024-09-14T22:56:09.836259+0000","last_clean":"2024-09-14T22:56:09.836259+0000","last_became_active":"2024-09-14T22:55:51.139441+0000","last_became_peered":"2024-09-14T22:55:51.139441+0000","last_unstale":"2024-09-14T22:56:09.836259+0000","last_undegraded":"2024-09-14T22:56:09.836259+0000","last_fullsized":"2024-09-14T22:56:09.836259+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:50.139115+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.026769095999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":10199,"num_read_kb":21422,"num_write":735,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837668+0000","last_change":"2024-09-14T22:55:32.040027+0000","last_active":"2024-09-14T22:56:09.837668+0000","last_peered":"2024-09-14T22:56:09.837668+0000","last_clean":"2024-09-14T22:56:09.837668+0000","last_became_active":"2024-09-14T22:55:32.039728+0000","last_became_peered":"2024-09-14T22:55:32.039728+0000","last_unstale":"2024-09-14T22:56:09.837668+0000","last_undegraded":"2024-09-14T22:56:09.837668+0000","last_fullsized":"2024-09-14T22:56:09.837668+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:36.095183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"170'487","reported_seq":6942,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314372+0000","last_change":"2024-09-14T22:55:51.267041+0000","last_active":"2024-09-14T22:56:10.314372+0000","last_peered":"2024-09-14T22:56:10.314372+0000","last_clean":"2024-09-14T22:56:10.314372+0000","last_became_active":"2024-09-14T22:55:51.141016+0000","last_became_peered":"2024-09-14T22:55:51.141016+0000","last_unstale":"2024-09-14T22:56:10.314372+0000","last_undegraded":"2024-09-14T22:56:10.314372+0000","last_fullsized":"2024-09-14T22:56:10.314372+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:00.580008+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.063591116000000003,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":6223,"num_read_kb":32196,"num_write":426,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313602+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:56:10.313602+0000","last_peered":"2024-09-14T22:56:10.313602+0000","last_clean":"2024-09-14T22:56:10.313602+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:56:10.313602+0000","last_undegraded":"2024-09-14T22:56:10.313602+0000","last_fullsized":"2024-09-14T22:56:10.313602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":503,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837732+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:56:09.837732+0000","last_peered":"2024-09-14T22:56:09.837732+0000","last_clean":"2024-09-14T22:56:09.837732+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:56:09.837732+0000","last_undegraded":"2024-09-14T22:56:09.837732+0000","last_fullsized":"2024-09-14T22:56:09.837732+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'940","reported_seq":10279,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312917+0000","last_change":"2024-09-14T22:55:36.056644+0000","last_active":"2024-09-14T22:56:10.312917+0000","last_peered":"2024-09-14T22:56:10.312917+0000","last_clean":"2024-09-14T22:56:10.312917+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:56:10.312917+0000","last_undegraded":"2024-09-14T22:56:10.312917+0000","last_fullsized":"2024-09-14T22:56:10.312917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:54.515748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019587290000000002,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":14513,"num_read_kb":36136,"num_write":948,"num_write_kb":13116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'531","reported_seq":2277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314277+0000","last_change":"2024-09-14T22:55:51.198762+0000","last_active":"2024-09-14T22:56:10.314277+0000","last_peered":"2024-09-14T22:56:10.314277+0000","last_clean":"2024-09-14T22:56:10.314277+0000","last_became_active":"2024-09-14T22:55:51.135840+0000","last_became_peered":"2024-09-14T22:55:51.135840+0000","last_unstale":"2024-09-14T22:56:10.314277+0000","last_undegraded":"2024-09-14T22:56:10.314277+0000","last_fullsized":"2024-09-14T22:56:10.314277+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:51.299112+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.046042974,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2706,"num_read_kb":9423,"num_write":453,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"170'531","reported_seq":2506,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313894+0000","last_change":"2024-09-14T22:55:51.296897+0000","last_active":"2024-09-14T22:56:10.313894+0000","last_peered":"2024-09-14T22:56:10.313894+0000","last_clean":"2024-09-14T22:56:10.313894+0000","last_became_active":"2024-09-14T22:55:51.143629+0000","last_became_peered":"2024-09-14T22:55:51.143629+0000","last_unstale":"2024-09-14T22:56:10.313894+0000","last_undegraded":"2024-09-14T22:56:10.313894+0000","last_fullsized":"2024-09-14T22:56:10.313894+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:51.928400+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.037412189999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":36,"num_read":3234,"num_read_kb":8551,"num_write":522,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'689","reported_seq":4144,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313011+0000","last_change":"2024-09-14T22:55:36.056675+0000","last_active":"2024-09-14T22:56:10.313011+0000","last_peered":"2024-09-14T22:56:10.313011+0000","last_clean":"2024-09-14T22:56:10.313011+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:56:10.313011+0000","last_undegraded":"2024-09-14T22:56:10.313011+0000","last_fullsized":"2024-09-14T22:56:10.313011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:48:33.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001452467,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5954,"num_read_kb":20565,"num_write":618,"num_write_kb":24613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'577","reported_seq":5751,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313749+0000","last_change":"2024-09-14T22:55:36.057396+0000","last_active":"2024-09-14T22:56:10.313749+0000","last_peered":"2024-09-14T22:56:10.313749+0000","last_clean":"2024-09-14T22:56:10.313749+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:56:10.313749+0000","last_undegraded":"2024-09-14T22:56:10.313749+0000","last_fullsized":"2024-09-14T22:56:10.313749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:58.321946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000939155,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":6243,"num_read_kb":22048,"num_write":553,"num_write_kb":16971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"162'626","reported_seq":7317,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837797+0000","last_change":"2024-09-14T22:55:36.135696+0000","last_active":"2024-09-14T22:56:09.837797+0000","last_peered":"2024-09-14T22:56:09.837797+0000","last_clean":"2024-09-14T22:56:09.837797+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:56:09.837797+0000","last_undegraded":"2024-09-14T22:56:09.837797+0000","last_fullsized":"2024-09-14T22:56:09.837797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:57.110923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014953080000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":8194,"num_read_kb":36336,"num_write":770,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"170'515","reported_seq":7151,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.838025+0000","last_change":"2024-09-14T22:55:51.374445+0000","last_active":"2024-09-14T22:56:09.838025+0000","last_peered":"2024-09-14T22:56:09.838025+0000","last_clean":"2024-09-14T22:56:09.838025+0000","last_became_active":"2024-09-14T22:55:51.141871+0000","last_became_peered":"2024-09-14T22:55:51.141871+0000","last_unstale":"2024-09-14T22:56:09.838025+0000","last_undegraded":"2024-09-14T22:56:09.838025+0000","last_fullsized":"2024-09-14T22:56:09.838025+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:57:25.784874+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.053472498,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":7727,"num_read_kb":40328,"num_write":448,"num_write_kb":21333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"170'348","reported_seq":6502,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837958+0000","last_change":"2024-09-14T22:55:51.268238+0000","last_active":"2024-09-14T22:56:09.837958+0000","last_peered":"2024-09-14T22:56:09.837958+0000","last_clean":"2024-09-14T22:56:09.837958+0000","last_became_active":"2024-09-14T22:55:51.141931+0000","last_became_peered":"2024-09-14T22:55:51.141931+0000","last_unstale":"2024-09-14T22:56:09.837958+0000","last_undegraded":"2024-09-14T22:56:09.837958+0000","last_fullsized":"2024-09-14T22:56:09.837958+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:22.125200+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.047107007999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5750,"num_read_kb":21575,"num_write":279,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'550","reported_seq":10161,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313774+0000","last_change":"2024-09-14T22:55:36.062330+0000","last_active":"2024-09-14T22:56:10.313774+0000","last_peered":"2024-09-14T22:56:10.313774+0000","last_clean":"2024-09-14T22:56:10.313774+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:56:10.313774+0000","last_undegraded":"2024-09-14T22:56:10.313774+0000","last_fullsized":"2024-09-14T22:56:10.313774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:02.042762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128364,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":11616,"num_read_kb":30610,"num_write":481,"num_write_kb":42134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"170'613","reported_seq":5711,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837789+0000","last_change":"2024-09-14T22:55:51.388404+0000","last_active":"2024-09-14T22:56:09.837789+0000","last_peered":"2024-09-14T22:56:09.837789+0000","last_clean":"2024-09-14T22:56:09.837789+0000","last_became_active":"2024-09-14T22:55:51.142779+0000","last_became_peered":"2024-09-14T22:55:51.142779+0000","last_unstale":"2024-09-14T22:56:09.837789+0000","last_undegraded":"2024-09-14T22:56:09.837789+0000","last_fullsized":"2024-09-14T22:56:09.837789+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:34.034047+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.041728379000000003,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":6204,"num_read_kb":34738,"num_write":712,"num_write_kb":24203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"161'410","reported_seq":1626,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836322+0000","last_change":"2024-09-14T22:55:56.769064+0000","last_active":"2024-09-14T22:56:09.836322+0000","last_peered":"2024-09-14T22:56:09.836322+0000","last_clean":"2024-09-14T22:56:09.836322+0000","last_became_active":"2024-09-14T22:55:56.767773+0000","last_became_peered":"2024-09-14T22:55:56.767773+0000","last_unstale":"2024-09-14T22:56:09.836322+0000","last_undegraded":"2024-09-14T22:56:09.836322+0000","last_fullsized":"2024-09-14T22:56:09.836322+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:58.130702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013471519999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":1274,"num_read_kb":10604,"num_write":428,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"170'983","reported_seq":6790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837827+0000","last_change":"2024-09-14T22:55:51.397026+0000","last_active":"2024-09-14T22:56:09.837827+0000","last_peered":"2024-09-14T22:56:09.837827+0000","last_clean":"2024-09-14T22:56:09.837827+0000","last_became_active":"2024-09-14T22:55:51.143935+0000","last_became_peered":"2024-09-14T22:55:51.143935+0000","last_unstale":"2024-09-14T22:56:09.837827+0000","last_undegraded":"2024-09-14T22:56:09.837827+0000","last_fullsized":"2024-09-14T22:56:09.837827+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:40:42.066436+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034758323000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":8863,"num_read_kb":12560,"num_write":1145,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":1377,"num_read":260757,"num_read_kb":1066701,"num_write":19012,"num_write_kb":702896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":141629825,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},"log_size":18820,"ondisk_log_size":18820,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56204,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315636,"statfs":{"total":96636764160,"available":96579211264,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29252,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342588,"statfs":{"total":96636764160,"available":96606810112,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51460,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320380,"statfs":{"total":96636764160,"available":96584069120,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":169,"seq":725849473030,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35772,"kb_used_data":2216,"kb_used_omap":47,"kb_used_meta":33488,"kb_avail":94336068,"statfs":{"total":96636764160,"available":96600133632,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49036,"internal_metadata":34291828},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2265088,"data_stored":1107477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41732,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330108,"statfs":{"total":96636764160,"available":96594030592,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29264,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342576,"statfs":{"total":96636764160,"available":96606797824,"internally_reserved":0,"allocated":2285568,"data_stored":1113560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56208,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315632,"statfs":{"total":96636764160,"available":96579207168,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:56:11.655 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:56:11.931 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:11.932 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:56:11.943 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-14T22:56:10.819047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1385,"num_object_clones":0,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":1377,"num_read":260873,"num_read_kb":1066801,"num_write":19173,"num_write_kb":705314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":141629863,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18940,"ondisk_log_size":18940,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333300,"kb_used_data":18932,"kb_used_omap":58,"kb_used_meta":314117,"kb_avail":754641420,"statfs":{"total":773094113280,"available":772752814080,"internally_reserved":0,"allocated":19386368,"data_stored":10070853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60192,"internal_metadata":321656032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000849"},"pg_stats":[{"pgid":"3.1f","version":"162'679","reported_seq":8427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836344+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836344+0000","last_peered":"2024-09-14T22:56:09.836344+0000","last_clean":"2024-09-14T22:56:09.836344+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:56:09.836344+0000","last_undegraded":"2024-09-14T22:56:09.836344+0000","last_fullsized":"2024-09-14T22:56:09.836344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:24:55.707066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001547748,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":9382,"num_read_kb":31310,"num_write":714,"num_write_kb":7692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"160'529","reported_seq":8618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313112+0000","last_change":"2024-09-14T22:55:36.055650+0000","last_active":"2024-09-14T22:56:10.313112+0000","last_peered":"2024-09-14T22:56:10.313112+0000","last_clean":"2024-09-14T22:56:10.313112+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:56:10.313112+0000","last_undegraded":"2024-09-14T22:56:10.313112+0000","last_fullsized":"2024-09-14T22:56:10.313112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:03:40.697937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012941300000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8409,"num_read_kb":20232,"num_write":463,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"170'521","reported_seq":9286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313783+0000","last_change":"2024-09-14T22:55:51.254209+0000","last_active":"2024-09-14T22:56:10.313783+0000","last_peered":"2024-09-14T22:56:10.313783+0000","last_clean":"2024-09-14T22:56:10.313783+0000","last_became_active":"2024-09-14T22:55:51.142689+0000","last_became_peered":"2024-09-14T22:55:51.142689+0000","last_unstale":"2024-09-14T22:56:10.313783+0000","last_undegraded":"2024-09-14T22:56:10.313783+0000","last_fullsized":"2024-09-14T22:56:10.313783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:02.520144+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.057378967000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9221,"num_read_kb":38644,"num_write":441,"num_write_kb":52016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"170'649","reported_seq":3618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313632+0000","last_change":"2024-09-14T22:55:51.339722+0000","last_active":"2024-09-14T22:56:10.313632+0000","last_peered":"2024-09-14T22:56:10.313632+0000","last_clean":"2024-09-14T22:56:10.313632+0000","last_became_active":"2024-09-14T22:55:51.144955+0000","last_became_peered":"2024-09-14T22:55:51.144955+0000","last_unstale":"2024-09-14T22:56:10.313632+0000","last_undegraded":"2024-09-14T22:56:10.313632+0000","last_fullsized":"2024-09-14T22:56:10.313632+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:33:11.938624+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.037609783000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":4493,"num_read_kb":12183,"num_write":718,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"170'340","reported_seq":5375,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836439+0000","last_change":"2024-09-14T22:55:51.202792+0000","last_active":"2024-09-14T22:56:09.836439+0000","last_peered":"2024-09-14T22:56:09.836439+0000","last_clean":"2024-09-14T22:56:09.836439+0000","last_became_active":"2024-09-14T22:55:51.137980+0000","last_became_peered":"2024-09-14T22:55:51.137980+0000","last_unstale":"2024-09-14T22:56:09.836439+0000","last_undegraded":"2024-09-14T22:56:09.836439+0000","last_fullsized":"2024-09-14T22:56:09.836439+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:12.182377+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034914117000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":4616,"num_read_kb":54005,"num_write":312,"num_write_kb":18747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"155'691","reported_seq":14554,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314213+0000","last_change":"2024-09-14T22:55:53.767098+0000","last_active":"2024-09-14T22:56:10.314213+0000","last_peered":"2024-09-14T22:56:10.314213+0000","last_clean":"2024-09-14T22:56:10.314213+0000","last_became_active":"2024-09-14T22:55:53.765594+0000","last_became_peered":"2024-09-14T22:55:53.765594+0000","last_unstale":"2024-09-14T22:56:10.314213+0000","last_undegraded":"2024-09-14T22:56:10.314213+0000","last_fullsized":"2024-09-14T22:56:10.314213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:56.211691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071064000000000001,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":16209,"num_read_kb":53499,"num_write":537,"num_write_kb":29099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":18306298,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"160'485","reported_seq":16175,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835713+0000","last_change":"2024-09-14T22:55:58.780115+0000","last_active":"2024-09-14T22:56:09.835713+0000","last_peered":"2024-09-14T22:56:09.835713+0000","last_clean":"2024-09-14T22:56:09.835713+0000","last_became_active":"2024-09-14T22:55:58.778624+0000","last_became_peered":"2024-09-14T22:55:58.778624+0000","last_unstale":"2024-09-14T22:56:09.835713+0000","last_undegraded":"2024-09-14T22:56:09.835713+0000","last_fullsized":"2024-09-14T22:56:09.835713+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:43.711519+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059748170000000003,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":15953,"num_read_kb":92561,"num_write":414,"num_write_kb":38470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'675","reported_seq":2449,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312849+0000","last_change":"2024-09-14T22:55:36.056150+0000","last_active":"2024-09-14T22:56:10.312849+0000","last_peered":"2024-09-14T22:56:10.312849+0000","last_clean":"2024-09-14T22:56:10.312849+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:56:10.312849+0000","last_undegraded":"2024-09-14T22:56:10.312849+0000","last_fullsized":"2024-09-14T22:56:10.312849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:58.129978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001629828,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":2959,"num_read_kb":22740,"num_write":861,"num_write_kb":10481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"161'652","reported_seq":6924,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836420+0000","last_change":"2024-09-14T22:55:56.769135+0000","last_active":"2024-09-14T22:56:09.836420+0000","last_peered":"2024-09-14T22:56:09.836420+0000","last_clean":"2024-09-14T22:56:09.836420+0000","last_became_active":"2024-09-14T22:55:56.768221+0000","last_became_peered":"2024-09-14T22:55:56.768221+0000","last_unstale":"2024-09-14T22:56:09.836420+0000","last_undegraded":"2024-09-14T22:56:09.836420+0000","last_fullsized":"2024-09-14T22:56:09.836420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:42.862372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001569408,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":7874,"num_read_kb":34799,"num_write":817,"num_write_kb":27727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"161'433","reported_seq":9778,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313725+0000","last_change":"2024-09-14T22:55:36.057943+0000","last_active":"2024-09-14T22:56:10.313725+0000","last_peered":"2024-09-14T22:56:10.313725+0000","last_clean":"2024-09-14T22:56:10.313725+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:56:10.313725+0000","last_undegraded":"2024-09-14T22:56:10.313725+0000","last_fullsized":"2024-09-14T22:56:10.313725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:01.697087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011312640000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10352,"num_read_kb":46008,"num_write":347,"num_write_kb":32677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"161'421","reported_seq":9422,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837884+0000","last_change":"2024-09-14T22:55:36.135035+0000","last_active":"2024-09-14T22:56:09.837884+0000","last_peered":"2024-09-14T22:56:09.837884+0000","last_clean":"2024-09-14T22:56:09.837884+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:56:09.837884+0000","last_undegraded":"2024-09-14T22:56:09.837884+0000","last_fullsized":"2024-09-14T22:56:09.837884+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:30.224048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086910600000000002,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10004,"num_read_kb":43389,"num_write":402,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"170'601","reported_seq":10915,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314273+0000","last_change":"2024-09-14T22:55:51.257123+0000","last_active":"2024-09-14T22:56:10.314273+0000","last_peered":"2024-09-14T22:56:10.314273+0000","last_clean":"2024-09-14T22:56:10.314273+0000","last_became_active":"2024-09-14T22:55:51.138619+0000","last_became_peered":"2024-09-14T22:55:51.138619+0000","last_unstale":"2024-09-14T22:56:10.314273+0000","last_undegraded":"2024-09-14T22:56:10.314273+0000","last_fullsized":"2024-09-14T22:56:10.314273+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:13.451285+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.074291557999999994,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11380,"num_read_kb":66402,"num_write":623,"num_write_kb":37521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313940+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:56:10.313940+0000","last_peered":"2024-09-14T22:56:10.313940+0000","last_clean":"2024-09-14T22:56:10.313940+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:56:10.313940+0000","last_undegraded":"2024-09-14T22:56:10.313940+0000","last_fullsized":"2024-09-14T22:56:10.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"162'728","reported_seq":8066,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836307+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836307+0000","last_peered":"2024-09-14T22:56:09.836307+0000","last_clean":"2024-09-14T22:56:09.836307+0000","last_became_active":"2024-09-14T22:55:32.040243+0000","last_became_peered":"2024-09-14T22:55:32.040243+0000","last_unstale":"2024-09-14T22:56:09.836307+0000","last_undegraded":"2024-09-14T22:56:09.836307+0000","last_fullsized":"2024-09-14T22:56:09.836307+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:28:24.384824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012928849999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":9421,"num_read_kb":42199,"num_write":751,"num_write_kb":24667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":310,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313779+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:56:10.313779+0000","last_peered":"2024-09-14T22:56:10.313779+0000","last_clean":"2024-09-14T22:56:10.313779+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:56:10.313779+0000","last_undegraded":"2024-09-14T22:56:10.313779+0000","last_fullsized":"2024-09-14T22:56:10.313779+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"156'335","reported_seq":2125,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837835+0000","last_change":"2024-09-14T22:55:36.134620+0000","last_active":"2024-09-14T22:56:09.837835+0000","last_peered":"2024-09-14T22:56:09.837835+0000","last_clean":"2024-09-14T22:56:09.837835+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:56:09.837835+0000","last_undegraded":"2024-09-14T22:56:09.837835+0000","last_fullsized":"2024-09-14T22:56:09.837835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:44.383091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079363000000000003,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":1234,"num_read_kb":15244,"num_write":279,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837945+0000","last_change":"2024-09-14T22:55:39.451745+0000","last_active":"2024-09-14T22:56:09.837945+0000","last_peered":"2024-09-14T22:56:09.837945+0000","last_clean":"2024-09-14T22:56:09.837945+0000","last_became_active":"2024-09-14T22:55:39.451421+0000","last_became_peered":"2024-09-14T22:55:39.451421+0000","last_unstale":"2024-09-14T22:56:09.837945+0000","last_undegraded":"2024-09-14T22:56:09.837945+0000","last_fullsized":"2024-09-14T22:56:09.837945+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:48.524842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"161'844","reported_seq":14297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835861+0000","last_change":"2024-09-14T22:55:58.778184+0000","last_active":"2024-09-14T22:56:09.835861+0000","last_peered":"2024-09-14T22:56:09.835861+0000","last_clean":"2024-09-14T22:56:09.835861+0000","last_became_active":"2024-09-14T22:55:58.776278+0000","last_became_peered":"2024-09-14T22:55:58.776278+0000","last_unstale":"2024-09-14T22:56:09.835861+0000","last_undegraded":"2024-09-14T22:56:09.835861+0000","last_fullsized":"2024-09-14T22:56:09.835861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:19:29.875793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992360000000002,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":15582,"num_read_kb":40098,"num_write":1081,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":15786495,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835949+0000","last_change":"2024-09-14T22:55:58.776581+0000","last_active":"2024-09-14T22:56:09.835949+0000","last_peered":"2024-09-14T22:56:09.835949+0000","last_clean":"2024-09-14T22:56:09.835949+0000","last_became_active":"2024-09-14T22:55:58.776284+0000","last_became_peered":"2024-09-14T22:55:58.776284+0000","last_unstale":"2024-09-14T22:56:09.835949+0000","last_undegraded":"2024-09-14T22:56:09.835949+0000","last_fullsized":"2024-09-14T22:56:09.835949+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:35.328965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"161'434","reported_seq":15790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313853+0000","last_change":"2024-09-14T22:55:36.057245+0000","last_active":"2024-09-14T22:56:10.313853+0000","last_peered":"2024-09-14T22:56:10.313853+0000","last_clean":"2024-09-14T22:56:10.313853+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:56:10.313853+0000","last_undegraded":"2024-09-14T22:56:10.313853+0000","last_fullsized":"2024-09-14T22:56:10.313853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:39:45.146222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055810799999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14507,"num_read_kb":48865,"num_write":357,"num_write_kb":19125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"162'1001","reported_seq":6842,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312863+0000","last_change":"2024-09-14T22:55:36.109343+0000","last_active":"2024-09-14T22:56:10.312863+0000","last_peered":"2024-09-14T22:56:10.312863+0000","last_clean":"2024-09-14T22:56:10.312863+0000","last_became_active":"2024-09-14T22:54:47.747387+0000","last_became_peered":"2024-09-14T22:54:47.747387+0000","last_unstale":"2024-09-14T22:56:10.312863+0000","last_undegraded":"2024-09-14T22:56:10.312863+0000","last_fullsized":"2024-09-14T22:56:10.312863+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:27.684254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356700000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":9071,"num_read_kb":57942,"num_write":1041,"num_write_kb":41152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":29933802,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313688+0000","last_change":"2024-09-14T22:55:32.043590+0000","last_active":"2024-09-14T22:56:10.313688+0000","last_peered":"2024-09-14T22:56:10.313688+0000","last_clean":"2024-09-14T22:56:10.313688+0000","last_became_active":"2024-09-14T22:55:32.040648+0000","last_became_peered":"2024-09-14T22:55:32.040648+0000","last_unstale":"2024-09-14T22:56:10.313688+0000","last_undegraded":"2024-09-14T22:56:10.313688+0000","last_fullsized":"2024-09-14T22:56:10.313688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:45:24.150360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"161'370","reported_seq":1908,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313752+0000","last_change":"2024-09-14T22:55:36.062326+0000","last_active":"2024-09-14T22:56:10.313752+0000","last_peered":"2024-09-14T22:56:10.313752+0000","last_clean":"2024-09-14T22:56:10.313752+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:56:10.313752+0000","last_undegraded":"2024-09-14T22:56:10.313752+0000","last_fullsized":"2024-09-14T22:56:10.313752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:43:31.326081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140828,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":1390,"num_read_kb":9489,"num_write":336,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":363,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312875+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:56:10.312875+0000","last_peered":"2024-09-14T22:56:10.312875+0000","last_clean":"2024-09-14T22:56:10.312875+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:56:10.312875+0000","last_undegraded":"2024-09-14T22:56:10.312875+0000","last_fullsized":"2024-09-14T22:56:10.312875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"170'632","reported_seq":9392,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836259+0000","last_change":"2024-09-14T22:55:51.310755+0000","last_active":"2024-09-14T22:56:09.836259+0000","last_peered":"2024-09-14T22:56:09.836259+0000","last_clean":"2024-09-14T22:56:09.836259+0000","last_became_active":"2024-09-14T22:55:51.139441+0000","last_became_peered":"2024-09-14T22:55:51.139441+0000","last_unstale":"2024-09-14T22:56:09.836259+0000","last_undegraded":"2024-09-14T22:56:09.836259+0000","last_fullsized":"2024-09-14T22:56:09.836259+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:50.139115+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.026769095999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":10199,"num_read_kb":21422,"num_write":735,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837668+0000","last_change":"2024-09-14T22:55:32.040027+0000","last_active":"2024-09-14T22:56:09.837668+0000","last_peered":"2024-09-14T22:56:09.837668+0000","last_clean":"2024-09-14T22:56:09.837668+0000","last_became_active":"2024-09-14T22:55:32.039728+0000","last_became_peered":"2024-09-14T22:55:32.039728+0000","last_unstale":"2024-09-14T22:56:09.837668+0000","last_undegraded":"2024-09-14T22:56:09.837668+0000","last_fullsized":"2024-09-14T22:56:09.837668+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:36.095183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"170'487","reported_seq":6942,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314372+0000","last_change":"2024-09-14T22:55:51.267041+0000","last_active":"2024-09-14T22:56:10.314372+0000","last_peered":"2024-09-14T22:56:10.314372+0000","last_clean":"2024-09-14T22:56:10.314372+0000","last_became_active":"2024-09-14T22:55:51.141016+0000","last_became_peered":"2024-09-14T22:55:51.141016+0000","last_unstale":"2024-09-14T22:56:10.314372+0000","last_undegraded":"2024-09-14T22:56:10.314372+0000","last_fullsized":"2024-09-14T22:56:10.314372+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:00.580008+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.063591116000000003,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":6223,"num_read_kb":32196,"num_write":426,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313602+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:56:10.313602+0000","last_peered":"2024-09-14T22:56:10.313602+0000","last_clean":"2024-09-14T22:56:10.313602+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:56:10.313602+0000","last_undegraded":"2024-09-14T22:56:10.313602+0000","last_fullsized":"2024-09-14T22:56:10.313602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":503,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837732+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:56:09.837732+0000","last_peered":"2024-09-14T22:56:09.837732+0000","last_clean":"2024-09-14T22:56:09.837732+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:56:09.837732+0000","last_undegraded":"2024-09-14T22:56:09.837732+0000","last_fullsized":"2024-09-14T22:56:09.837732+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'940","reported_seq":10279,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312917+0000","last_change":"2024-09-14T22:55:36.056644+0000","last_active":"2024-09-14T22:56:10.312917+0000","last_peered":"2024-09-14T22:56:10.312917+0000","last_clean":"2024-09-14T22:56:10.312917+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:56:10.312917+0000","last_undegraded":"2024-09-14T22:56:10.312917+0000","last_fullsized":"2024-09-14T22:56:10.312917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:54.515748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019587290000000002,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":14513,"num_read_kb":36136,"num_write":948,"num_write_kb":13116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'531","reported_seq":2277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314277+0000","last_change":"2024-09-14T22:55:51.198762+0000","last_active":"2024-09-14T22:56:10.314277+0000","last_peered":"2024-09-14T22:56:10.314277+0000","last_clean":"2024-09-14T22:56:10.314277+0000","last_became_active":"2024-09-14T22:55:51.135840+0000","last_became_peered":"2024-09-14T22:55:51.135840+0000","last_unstale":"2024-09-14T22:56:10.314277+0000","last_undegraded":"2024-09-14T22:56:10.314277+0000","last_fullsized":"2024-09-14T22:56:10.314277+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:51.299112+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.046042974,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2706,"num_read_kb":9423,"num_write":453,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"170'531","reported_seq":2506,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313894+0000","last_change":"2024-09-14T22:55:51.296897+0000","last_active":"2024-09-14T22:56:10.313894+0000","last_peered":"2024-09-14T22:56:10.313894+0000","last_clean":"2024-09-14T22:56:10.313894+0000","last_became_active":"2024-09-14T22:55:51.143629+0000","last_became_peered":"2024-09-14T22:55:51.143629+0000","last_unstale":"2024-09-14T22:56:10.313894+0000","last_undegraded":"2024-09-14T22:56:10.313894+0000","last_fullsized":"2024-09-14T22:56:10.313894+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:51.928400+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.037412189999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":36,"num_read":3234,"num_read_kb":8551,"num_write":522,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'689","reported_seq":4144,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313011+0000","last_change":"2024-09-14T22:55:36.056675+0000","last_active":"2024-09-14T22:56:10.313011+0000","last_peered":"2024-09-14T22:56:10.313011+0000","last_clean":"2024-09-14T22:56:10.313011+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:56:10.313011+0000","last_undegraded":"2024-09-14T22:56:10.313011+0000","last_fullsized":"2024-09-14T22:56:10.313011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:48:33.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001452467,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5954,"num_read_kb":20565,"num_write":618,"num_write_kb":24613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'577","reported_seq":5751,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313749+0000","last_change":"2024-09-14T22:55:36.057396+0000","last_active":"2024-09-14T22:56:10.313749+0000","last_peered":"2024-09-14T22:56:10.313749+0000","last_clean":"2024-09-14T22:56:10.313749+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:56:10.313749+0000","last_undegraded":"2024-09-14T22:56:10.313749+0000","last_fullsized":"2024-09-14T22:56:10.313749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:58.321946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000939155,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":6243,"num_read_kb":22048,"num_write":553,"num_write_kb":16971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"162'626","reported_seq":7317,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837797+0000","last_change":"2024-09-14T22:55:36.135696+0000","last_active":"2024-09-14T22:56:09.837797+0000","last_peered":"2024-09-14T22:56:09.837797+0000","last_clean":"2024-09-14T22:56:09.837797+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:56:09.837797+0000","last_undegraded":"2024-09-14T22:56:09.837797+0000","last_fullsized":"2024-09-14T22:56:09.837797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:57.110923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014953080000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":8194,"num_read_kb":36336,"num_write":770,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"170'515","reported_seq":7151,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.838025+0000","last_change":"2024-09-14T22:55:51.374445+0000","last_active":"2024-09-14T22:56:09.838025+0000","last_peered":"2024-09-14T22:56:09.838025+0000","last_clean":"2024-09-14T22:56:09.838025+0000","last_became_active":"2024-09-14T22:55:51.141871+0000","last_became_peered":"2024-09-14T22:55:51.141871+0000","last_unstale":"2024-09-14T22:56:09.838025+0000","last_undegraded":"2024-09-14T22:56:09.838025+0000","last_fullsized":"2024-09-14T22:56:09.838025+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:57:25.784874+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.053472498,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":7727,"num_read_kb":40328,"num_write":448,"num_write_kb":21333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"170'348","reported_seq":6502,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837958+0000","last_change":"2024-09-14T22:55:51.268238+0000","last_active":"2024-09-14T22:56:09.837958+0000","last_peered":"2024-09-14T22:56:09.837958+0000","last_clean":"2024-09-14T22:56:09.837958+0000","last_became_active":"2024-09-14T22:55:51.141931+0000","last_became_peered":"2024-09-14T22:55:51.141931+0000","last_unstale":"2024-09-14T22:56:09.837958+0000","last_undegraded":"2024-09-14T22:56:09.837958+0000","last_fullsized":"2024-09-14T22:56:09.837958+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:22.125200+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.047107007999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5750,"num_read_kb":21575,"num_write":279,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'550","reported_seq":10161,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313774+0000","last_change":"2024-09-14T22:55:36.062330+0000","last_active":"2024-09-14T22:56:10.313774+0000","last_peered":"2024-09-14T22:56:10.313774+0000","last_clean":"2024-09-14T22:56:10.313774+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:56:10.313774+0000","last_undegraded":"2024-09-14T22:56:10.313774+0000","last_fullsized":"2024-09-14T22:56:10.313774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:02.042762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128364,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":11616,"num_read_kb":30610,"num_write":481,"num_write_kb":42134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"170'613","reported_seq":5711,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837789+0000","last_change":"2024-09-14T22:55:51.388404+0000","last_active":"2024-09-14T22:56:09.837789+0000","last_peered":"2024-09-14T22:56:09.837789+0000","last_clean":"2024-09-14T22:56:09.837789+0000","last_became_active":"2024-09-14T22:55:51.142779+0000","last_became_peered":"2024-09-14T22:55:51.142779+0000","last_unstale":"2024-09-14T22:56:09.837789+0000","last_undegraded":"2024-09-14T22:56:09.837789+0000","last_fullsized":"2024-09-14T22:56:09.837789+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:34.034047+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.041728379000000003,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":6204,"num_read_kb":34738,"num_write":712,"num_write_kb":24203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"161'410","reported_seq":1626,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836322+0000","last_change":"2024-09-14T22:55:56.769064+0000","last_active":"2024-09-14T22:56:09.836322+0000","last_peered":"2024-09-14T22:56:09.836322+0000","last_clean":"2024-09-14T22:56:09.836322+0000","last_became_active":"2024-09-14T22:55:56.767773+0000","last_became_peered":"2024-09-14T22:55:56.767773+0000","last_unstale":"2024-09-14T22:56:09.836322+0000","last_undegraded":"2024-09-14T22:56:09.836322+0000","last_fullsized":"2024-09-14T22:56:09.836322+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:58.130702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013471519999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":1274,"num_read_kb":10604,"num_write":428,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"170'983","reported_seq":6790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837827+0000","last_change":"2024-09-14T22:55:51.397026+0000","last_active":"2024-09-14T22:56:09.837827+0000","last_peered":"2024-09-14T22:56:09.837827+0000","last_clean":"2024-09-14T22:56:09.837827+0000","last_became_active":"2024-09-14T22:55:51.143935+0000","last_became_peered":"2024-09-14T22:55:51.143935+0000","last_unstale":"2024-09-14T22:56:09.837827+0000","last_undegraded":"2024-09-14T22:56:09.837827+0000","last_fullsized":"2024-09-14T22:56:09.837827+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:40:42.066436+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034758323000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":8863,"num_read_kb":12560,"num_write":1145,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":1377,"num_read":260757,"num_read_kb":1066701,"num_write":19012,"num_write_kb":702896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":141629825,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},"log_size":18820,"ondisk_log_size":18820,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56204,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315636,"statfs":{"total":96636764160,"available":96579211264,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29252,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342588,"statfs":{"total":96636764160,"available":96606810112,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51460,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320380,"statfs":{"total":96636764160,"available":96584069120,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":169,"seq":725849473030,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35772,"kb_used_data":2216,"kb_used_omap":47,"kb_used_meta":33488,"kb_avail":94336068,"statfs":{"total":96636764160,"available":96600133632,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49036,"internal_metadata":34291828},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2265088,"data_stored":1107477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41732,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330108,"statfs":{"total":96636764160,"available":96594030592,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29264,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342576,"statfs":{"total":96636764160,"available":96606797824,"internally_reserved":0,"allocated":2285568,"data_stored":1113560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56208,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315632,"statfs":{"total":96636764160,"available":96579207168,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:56:11.944 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T22:56:11.945 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T22:56:12.238 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T22:56:12.250 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T22:56:12.251 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:56:12.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-14T22:56:12.553+0000 7f4dcffff640 0 --2- 172.21.15.76:0/982872540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4dd01427d0 0x7f4dd008bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T22:56:12.729 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:12.730 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:56:12.741 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-14T22:56:10.819047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1385,"num_object_clones":0,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":1377,"num_read":260873,"num_read_kb":1066801,"num_write":19173,"num_write_kb":705314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":141629863,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18940,"ondisk_log_size":18940,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333300,"kb_used_data":18932,"kb_used_omap":58,"kb_used_meta":314117,"kb_avail":754641420,"statfs":{"total":773094113280,"available":772752814080,"internally_reserved":0,"allocated":19386368,"data_stored":10070853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60192,"internal_metadata":321656032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000849"},"pg_stats":[{"pgid":"3.1f","version":"162'679","reported_seq":8427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836344+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836344+0000","last_peered":"2024-09-14T22:56:09.836344+0000","last_clean":"2024-09-14T22:56:09.836344+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:56:09.836344+0000","last_undegraded":"2024-09-14T22:56:09.836344+0000","last_fullsized":"2024-09-14T22:56:09.836344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:24:55.707066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001547748,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":9382,"num_read_kb":31310,"num_write":714,"num_write_kb":7692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"160'529","reported_seq":8618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313112+0000","last_change":"2024-09-14T22:55:36.055650+0000","last_active":"2024-09-14T22:56:10.313112+0000","last_peered":"2024-09-14T22:56:10.313112+0000","last_clean":"2024-09-14T22:56:10.313112+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:56:10.313112+0000","last_undegraded":"2024-09-14T22:56:10.313112+0000","last_fullsized":"2024-09-14T22:56:10.313112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:03:40.697937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012941300000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8409,"num_read_kb":20232,"num_write":463,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"170'521","reported_seq":9286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313783+0000","last_change":"2024-09-14T22:55:51.254209+0000","last_active":"2024-09-14T22:56:10.313783+0000","last_peered":"2024-09-14T22:56:10.313783+0000","last_clean":"2024-09-14T22:56:10.313783+0000","last_became_active":"2024-09-14T22:55:51.142689+0000","last_became_peered":"2024-09-14T22:55:51.142689+0000","last_unstale":"2024-09-14T22:56:10.313783+0000","last_undegraded":"2024-09-14T22:56:10.313783+0000","last_fullsized":"2024-09-14T22:56:10.313783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:02.520144+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.057378967000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9221,"num_read_kb":38644,"num_write":441,"num_write_kb":52016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"170'649","reported_seq":3618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313632+0000","last_change":"2024-09-14T22:55:51.339722+0000","last_active":"2024-09-14T22:56:10.313632+0000","last_peered":"2024-09-14T22:56:10.313632+0000","last_clean":"2024-09-14T22:56:10.313632+0000","last_became_active":"2024-09-14T22:55:51.144955+0000","last_became_peered":"2024-09-14T22:55:51.144955+0000","last_unstale":"2024-09-14T22:56:10.313632+0000","last_undegraded":"2024-09-14T22:56:10.313632+0000","last_fullsized":"2024-09-14T22:56:10.313632+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:33:11.938624+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.037609783000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":4493,"num_read_kb":12183,"num_write":718,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"170'340","reported_seq":5375,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836439+0000","last_change":"2024-09-14T22:55:51.202792+0000","last_active":"2024-09-14T22:56:09.836439+0000","last_peered":"2024-09-14T22:56:09.836439+0000","last_clean":"2024-09-14T22:56:09.836439+0000","last_became_active":"2024-09-14T22:55:51.137980+0000","last_became_peered":"2024-09-14T22:55:51.137980+0000","last_unstale":"2024-09-14T22:56:09.836439+0000","last_undegraded":"2024-09-14T22:56:09.836439+0000","last_fullsized":"2024-09-14T22:56:09.836439+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:12.182377+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034914117000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":4616,"num_read_kb":54005,"num_write":312,"num_write_kb":18747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"155'691","reported_seq":14554,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314213+0000","last_change":"2024-09-14T22:55:53.767098+0000","last_active":"2024-09-14T22:56:10.314213+0000","last_peered":"2024-09-14T22:56:10.314213+0000","last_clean":"2024-09-14T22:56:10.314213+0000","last_became_active":"2024-09-14T22:55:53.765594+0000","last_became_peered":"2024-09-14T22:55:53.765594+0000","last_unstale":"2024-09-14T22:56:10.314213+0000","last_undegraded":"2024-09-14T22:56:10.314213+0000","last_fullsized":"2024-09-14T22:56:10.314213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:56.211691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071064000000000001,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":16209,"num_read_kb":53499,"num_write":537,"num_write_kb":29099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":18306298,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"160'485","reported_seq":16175,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835713+0000","last_change":"2024-09-14T22:55:58.780115+0000","last_active":"2024-09-14T22:56:09.835713+0000","last_peered":"2024-09-14T22:56:09.835713+0000","last_clean":"2024-09-14T22:56:09.835713+0000","last_became_active":"2024-09-14T22:55:58.778624+0000","last_became_peered":"2024-09-14T22:55:58.778624+0000","last_unstale":"2024-09-14T22:56:09.835713+0000","last_undegraded":"2024-09-14T22:56:09.835713+0000","last_fullsized":"2024-09-14T22:56:09.835713+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:43.711519+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059748170000000003,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":15953,"num_read_kb":92561,"num_write":414,"num_write_kb":38470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'675","reported_seq":2449,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312849+0000","last_change":"2024-09-14T22:55:36.056150+0000","last_active":"2024-09-14T22:56:10.312849+0000","last_peered":"2024-09-14T22:56:10.312849+0000","last_clean":"2024-09-14T22:56:10.312849+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:56:10.312849+0000","last_undegraded":"2024-09-14T22:56:10.312849+0000","last_fullsized":"2024-09-14T22:56:10.312849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:58.129978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001629828,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":2959,"num_read_kb":22740,"num_write":861,"num_write_kb":10481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"161'652","reported_seq":6924,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836420+0000","last_change":"2024-09-14T22:55:56.769135+0000","last_active":"2024-09-14T22:56:09.836420+0000","last_peered":"2024-09-14T22:56:09.836420+0000","last_clean":"2024-09-14T22:56:09.836420+0000","last_became_active":"2024-09-14T22:55:56.768221+0000","last_became_peered":"2024-09-14T22:55:56.768221+0000","last_unstale":"2024-09-14T22:56:09.836420+0000","last_undegraded":"2024-09-14T22:56:09.836420+0000","last_fullsized":"2024-09-14T22:56:09.836420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:42.862372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001569408,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":7874,"num_read_kb":34799,"num_write":817,"num_write_kb":27727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"161'433","reported_seq":9778,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313725+0000","last_change":"2024-09-14T22:55:36.057943+0000","last_active":"2024-09-14T22:56:10.313725+0000","last_peered":"2024-09-14T22:56:10.313725+0000","last_clean":"2024-09-14T22:56:10.313725+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:56:10.313725+0000","last_undegraded":"2024-09-14T22:56:10.313725+0000","last_fullsized":"2024-09-14T22:56:10.313725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:01.697087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011312640000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10352,"num_read_kb":46008,"num_write":347,"num_write_kb":32677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"161'421","reported_seq":9422,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837884+0000","last_change":"2024-09-14T22:55:36.135035+0000","last_active":"2024-09-14T22:56:09.837884+0000","last_peered":"2024-09-14T22:56:09.837884+0000","last_clean":"2024-09-14T22:56:09.837884+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:56:09.837884+0000","last_undegraded":"2024-09-14T22:56:09.837884+0000","last_fullsized":"2024-09-14T22:56:09.837884+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:30.224048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086910600000000002,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10004,"num_read_kb":43389,"num_write":402,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"170'601","reported_seq":10915,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314273+0000","last_change":"2024-09-14T22:55:51.257123+0000","last_active":"2024-09-14T22:56:10.314273+0000","last_peered":"2024-09-14T22:56:10.314273+0000","last_clean":"2024-09-14T22:56:10.314273+0000","last_became_active":"2024-09-14T22:55:51.138619+0000","last_became_peered":"2024-09-14T22:55:51.138619+0000","last_unstale":"2024-09-14T22:56:10.314273+0000","last_undegraded":"2024-09-14T22:56:10.314273+0000","last_fullsized":"2024-09-14T22:56:10.314273+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:13.451285+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.074291557999999994,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11380,"num_read_kb":66402,"num_write":623,"num_write_kb":37521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313940+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:56:10.313940+0000","last_peered":"2024-09-14T22:56:10.313940+0000","last_clean":"2024-09-14T22:56:10.313940+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:56:10.313940+0000","last_undegraded":"2024-09-14T22:56:10.313940+0000","last_fullsized":"2024-09-14T22:56:10.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"162'728","reported_seq":8066,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836307+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836307+0000","last_peered":"2024-09-14T22:56:09.836307+0000","last_clean":"2024-09-14T22:56:09.836307+0000","last_became_active":"2024-09-14T22:55:32.040243+0000","last_became_peered":"2024-09-14T22:55:32.040243+0000","last_unstale":"2024-09-14T22:56:09.836307+0000","last_undegraded":"2024-09-14T22:56:09.836307+0000","last_fullsized":"2024-09-14T22:56:09.836307+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:28:24.384824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012928849999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":9421,"num_read_kb":42199,"num_write":751,"num_write_kb":24667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":310,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313779+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:56:10.313779+0000","last_peered":"2024-09-14T22:56:10.313779+0000","last_clean":"2024-09-14T22:56:10.313779+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:56:10.313779+0000","last_undegraded":"2024-09-14T22:56:10.313779+0000","last_fullsized":"2024-09-14T22:56:10.313779+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"156'335","reported_seq":2125,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837835+0000","last_change":"2024-09-14T22:55:36.134620+0000","last_active":"2024-09-14T22:56:09.837835+0000","last_peered":"2024-09-14T22:56:09.837835+0000","last_clean":"2024-09-14T22:56:09.837835+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:56:09.837835+0000","last_undegraded":"2024-09-14T22:56:09.837835+0000","last_fullsized":"2024-09-14T22:56:09.837835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:44.383091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079363000000000003,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":1234,"num_read_kb":15244,"num_write":279,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837945+0000","last_change":"2024-09-14T22:55:39.451745+0000","last_active":"2024-09-14T22:56:09.837945+0000","last_peered":"2024-09-14T22:56:09.837945+0000","last_clean":"2024-09-14T22:56:09.837945+0000","last_became_active":"2024-09-14T22:55:39.451421+0000","last_became_peered":"2024-09-14T22:55:39.451421+0000","last_unstale":"2024-09-14T22:56:09.837945+0000","last_undegraded":"2024-09-14T22:56:09.837945+0000","last_fullsized":"2024-09-14T22:56:09.837945+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:48.524842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"161'844","reported_seq":14297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835861+0000","last_change":"2024-09-14T22:55:58.778184+0000","last_active":"2024-09-14T22:56:09.835861+0000","last_peered":"2024-09-14T22:56:09.835861+0000","last_clean":"2024-09-14T22:56:09.835861+0000","last_became_active":"2024-09-14T22:55:58.776278+0000","last_became_peered":"2024-09-14T22:55:58.776278+0000","last_unstale":"2024-09-14T22:56:09.835861+0000","last_undegraded":"2024-09-14T22:56:09.835861+0000","last_fullsized":"2024-09-14T22:56:09.835861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:19:29.875793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992360000000002,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":15582,"num_read_kb":40098,"num_write":1081,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":15786495,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835949+0000","last_change":"2024-09-14T22:55:58.776581+0000","last_active":"2024-09-14T22:56:09.835949+0000","last_peered":"2024-09-14T22:56:09.835949+0000","last_clean":"2024-09-14T22:56:09.835949+0000","last_became_active":"2024-09-14T22:55:58.776284+0000","last_became_peered":"2024-09-14T22:55:58.776284+0000","last_unstale":"2024-09-14T22:56:09.835949+0000","last_undegraded":"2024-09-14T22:56:09.835949+0000","last_fullsized":"2024-09-14T22:56:09.835949+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:35.328965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"161'434","reported_seq":15790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313853+0000","last_change":"2024-09-14T22:55:36.057245+0000","last_active":"2024-09-14T22:56:10.313853+0000","last_peered":"2024-09-14T22:56:10.313853+0000","last_clean":"2024-09-14T22:56:10.313853+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:56:10.313853+0000","last_undegraded":"2024-09-14T22:56:10.313853+0000","last_fullsized":"2024-09-14T22:56:10.313853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:39:45.146222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055810799999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14507,"num_read_kb":48865,"num_write":357,"num_write_kb":19125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"162'1001","reported_seq":6842,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312863+0000","last_change":"2024-09-14T22:55:36.109343+0000","last_active":"2024-09-14T22:56:10.312863+0000","last_peered":"2024-09-14T22:56:10.312863+0000","last_clean":"2024-09-14T22:56:10.312863+0000","last_became_active":"2024-09-14T22:54:47.747387+0000","last_became_peered":"2024-09-14T22:54:47.747387+0000","last_unstale":"2024-09-14T22:56:10.312863+0000","last_undegraded":"2024-09-14T22:56:10.312863+0000","last_fullsized":"2024-09-14T22:56:10.312863+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:27.684254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356700000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":9071,"num_read_kb":57942,"num_write":1041,"num_write_kb":41152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":29933802,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313688+0000","last_change":"2024-09-14T22:55:32.043590+0000","last_active":"2024-09-14T22:56:10.313688+0000","last_peered":"2024-09-14T22:56:10.313688+0000","last_clean":"2024-09-14T22:56:10.313688+0000","last_became_active":"2024-09-14T22:55:32.040648+0000","last_became_peered":"2024-09-14T22:55:32.040648+0000","last_unstale":"2024-09-14T22:56:10.313688+0000","last_undegraded":"2024-09-14T22:56:10.313688+0000","last_fullsized":"2024-09-14T22:56:10.313688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:45:24.150360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"161'370","reported_seq":1908,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313752+0000","last_change":"2024-09-14T22:55:36.062326+0000","last_active":"2024-09-14T22:56:10.313752+0000","last_peered":"2024-09-14T22:56:10.313752+0000","last_clean":"2024-09-14T22:56:10.313752+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:56:10.313752+0000","last_undegraded":"2024-09-14T22:56:10.313752+0000","last_fullsized":"2024-09-14T22:56:10.313752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:43:31.326081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140828,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":1390,"num_read_kb":9489,"num_write":336,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":363,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312875+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:56:10.312875+0000","last_peered":"2024-09-14T22:56:10.312875+0000","last_clean":"2024-09-14T22:56:10.312875+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:56:10.312875+0000","last_undegraded":"2024-09-14T22:56:10.312875+0000","last_fullsized":"2024-09-14T22:56:10.312875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"170'632","reported_seq":9392,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836259+0000","last_change":"2024-09-14T22:55:51.310755+0000","last_active":"2024-09-14T22:56:09.836259+0000","last_peered":"2024-09-14T22:56:09.836259+0000","last_clean":"2024-09-14T22:56:09.836259+0000","last_became_active":"2024-09-14T22:55:51.139441+0000","last_became_peered":"2024-09-14T22:55:51.139441+0000","last_unstale":"2024-09-14T22:56:09.836259+0000","last_undegraded":"2024-09-14T22:56:09.836259+0000","last_fullsized":"2024-09-14T22:56:09.836259+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:50.139115+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.026769095999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":10199,"num_read_kb":21422,"num_write":735,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837668+0000","last_change":"2024-09-14T22:55:32.040027+0000","last_active":"2024-09-14T22:56:09.837668+0000","last_peered":"2024-09-14T22:56:09.837668+0000","last_clean":"2024-09-14T22:56:09.837668+0000","last_became_active":"2024-09-14T22:55:32.039728+0000","last_became_peered":"2024-09-14T22:55:32.039728+0000","last_unstale":"2024-09-14T22:56:09.837668+0000","last_undegraded":"2024-09-14T22:56:09.837668+0000","last_fullsized":"2024-09-14T22:56:09.837668+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:36.095183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"170'487","reported_seq":6942,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314372+0000","last_change":"2024-09-14T22:55:51.267041+0000","last_active":"2024-09-14T22:56:10.314372+0000","last_peered":"2024-09-14T22:56:10.314372+0000","last_clean":"2024-09-14T22:56:10.314372+0000","last_became_active":"2024-09-14T22:55:51.141016+0000","last_became_peered":"2024-09-14T22:55:51.141016+0000","last_unstale":"2024-09-14T22:56:10.314372+0000","last_undegraded":"2024-09-14T22:56:10.314372+0000","last_fullsized":"2024-09-14T22:56:10.314372+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:00.580008+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.063591116000000003,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":6223,"num_read_kb":32196,"num_write":426,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313602+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:56:10.313602+0000","last_peered":"2024-09-14T22:56:10.313602+0000","last_clean":"2024-09-14T22:56:10.313602+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:56:10.313602+0000","last_undegraded":"2024-09-14T22:56:10.313602+0000","last_fullsized":"2024-09-14T22:56:10.313602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":503,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837732+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:56:09.837732+0000","last_peered":"2024-09-14T22:56:09.837732+0000","last_clean":"2024-09-14T22:56:09.837732+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:56:09.837732+0000","last_undegraded":"2024-09-14T22:56:09.837732+0000","last_fullsized":"2024-09-14T22:56:09.837732+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'940","reported_seq":10279,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312917+0000","last_change":"2024-09-14T22:55:36.056644+0000","last_active":"2024-09-14T22:56:10.312917+0000","last_peered":"2024-09-14T22:56:10.312917+0000","last_clean":"2024-09-14T22:56:10.312917+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:56:10.312917+0000","last_undegraded":"2024-09-14T22:56:10.312917+0000","last_fullsized":"2024-09-14T22:56:10.312917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:54.515748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019587290000000002,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":14513,"num_read_kb":36136,"num_write":948,"num_write_kb":13116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'531","reported_seq":2277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314277+0000","last_change":"2024-09-14T22:55:51.198762+0000","last_active":"2024-09-14T22:56:10.314277+0000","last_peered":"2024-09-14T22:56:10.314277+0000","last_clean":"2024-09-14T22:56:10.314277+0000","last_became_active":"2024-09-14T22:55:51.135840+0000","last_became_peered":"2024-09-14T22:55:51.135840+0000","last_unstale":"2024-09-14T22:56:10.314277+0000","last_undegraded":"2024-09-14T22:56:10.314277+0000","last_fullsized":"2024-09-14T22:56:10.314277+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:51.299112+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.046042974,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2706,"num_read_kb":9423,"num_write":453,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"170'531","reported_seq":2506,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313894+0000","last_change":"2024-09-14T22:55:51.296897+0000","last_active":"2024-09-14T22:56:10.313894+0000","last_peered":"2024-09-14T22:56:10.313894+0000","last_clean":"2024-09-14T22:56:10.313894+0000","last_became_active":"2024-09-14T22:55:51.143629+0000","last_became_peered":"2024-09-14T22:55:51.143629+0000","last_unstale":"2024-09-14T22:56:10.313894+0000","last_undegraded":"2024-09-14T22:56:10.313894+0000","last_fullsized":"2024-09-14T22:56:10.313894+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:51.928400+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.037412189999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":36,"num_read":3234,"num_read_kb":8551,"num_write":522,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'689","reported_seq":4144,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313011+0000","last_change":"2024-09-14T22:55:36.056675+0000","last_active":"2024-09-14T22:56:10.313011+0000","last_peered":"2024-09-14T22:56:10.313011+0000","last_clean":"2024-09-14T22:56:10.313011+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:56:10.313011+0000","last_undegraded":"2024-09-14T22:56:10.313011+0000","last_fullsized":"2024-09-14T22:56:10.313011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:48:33.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001452467,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5954,"num_read_kb":20565,"num_write":618,"num_write_kb":24613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'577","reported_seq":5751,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313749+0000","last_change":"2024-09-14T22:55:36.057396+0000","last_active":"2024-09-14T22:56:10.313749+0000","last_peered":"2024-09-14T22:56:10.313749+0000","last_clean":"2024-09-14T22:56:10.313749+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:56:10.313749+0000","last_undegraded":"2024-09-14T22:56:10.313749+0000","last_fullsized":"2024-09-14T22:56:10.313749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:58.321946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000939155,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":6243,"num_read_kb":22048,"num_write":553,"num_write_kb":16971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"162'626","reported_seq":7317,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837797+0000","last_change":"2024-09-14T22:55:36.135696+0000","last_active":"2024-09-14T22:56:09.837797+0000","last_peered":"2024-09-14T22:56:09.837797+0000","last_clean":"2024-09-14T22:56:09.837797+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:56:09.837797+0000","last_undegraded":"2024-09-14T22:56:09.837797+0000","last_fullsized":"2024-09-14T22:56:09.837797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:57.110923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014953080000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":8194,"num_read_kb":36336,"num_write":770,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"170'515","reported_seq":7151,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.838025+0000","last_change":"2024-09-14T22:55:51.374445+0000","last_active":"2024-09-14T22:56:09.838025+0000","last_peered":"2024-09-14T22:56:09.838025+0000","last_clean":"2024-09-14T22:56:09.838025+0000","last_became_active":"2024-09-14T22:55:51.141871+0000","last_became_peered":"2024-09-14T22:55:51.141871+0000","last_unstale":"2024-09-14T22:56:09.838025+0000","last_undegraded":"2024-09-14T22:56:09.838025+0000","last_fullsized":"2024-09-14T22:56:09.838025+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:57:25.784874+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.053472498,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":7727,"num_read_kb":40328,"num_write":448,"num_write_kb":21333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"170'348","reported_seq":6502,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837958+0000","last_change":"2024-09-14T22:55:51.268238+0000","last_active":"2024-09-14T22:56:09.837958+0000","last_peered":"2024-09-14T22:56:09.837958+0000","last_clean":"2024-09-14T22:56:09.837958+0000","last_became_active":"2024-09-14T22:55:51.141931+0000","last_became_peered":"2024-09-14T22:55:51.141931+0000","last_unstale":"2024-09-14T22:56:09.837958+0000","last_undegraded":"2024-09-14T22:56:09.837958+0000","last_fullsized":"2024-09-14T22:56:09.837958+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:22.125200+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.047107007999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5750,"num_read_kb":21575,"num_write":279,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'550","reported_seq":10161,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313774+0000","last_change":"2024-09-14T22:55:36.062330+0000","last_active":"2024-09-14T22:56:10.313774+0000","last_peered":"2024-09-14T22:56:10.313774+0000","last_clean":"2024-09-14T22:56:10.313774+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:56:10.313774+0000","last_undegraded":"2024-09-14T22:56:10.313774+0000","last_fullsized":"2024-09-14T22:56:10.313774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:02.042762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128364,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":11616,"num_read_kb":30610,"num_write":481,"num_write_kb":42134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"170'613","reported_seq":5711,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837789+0000","last_change":"2024-09-14T22:55:51.388404+0000","last_active":"2024-09-14T22:56:09.837789+0000","last_peered":"2024-09-14T22:56:09.837789+0000","last_clean":"2024-09-14T22:56:09.837789+0000","last_became_active":"2024-09-14T22:55:51.142779+0000","last_became_peered":"2024-09-14T22:55:51.142779+0000","last_unstale":"2024-09-14T22:56:09.837789+0000","last_undegraded":"2024-09-14T22:56:09.837789+0000","last_fullsized":"2024-09-14T22:56:09.837789+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:34.034047+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.041728379000000003,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":6204,"num_read_kb":34738,"num_write":712,"num_write_kb":24203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"161'410","reported_seq":1626,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836322+0000","last_change":"2024-09-14T22:55:56.769064+0000","last_active":"2024-09-14T22:56:09.836322+0000","last_peered":"2024-09-14T22:56:09.836322+0000","last_clean":"2024-09-14T22:56:09.836322+0000","last_became_active":"2024-09-14T22:55:56.767773+0000","last_became_peered":"2024-09-14T22:55:56.767773+0000","last_unstale":"2024-09-14T22:56:09.836322+0000","last_undegraded":"2024-09-14T22:56:09.836322+0000","last_fullsized":"2024-09-14T22:56:09.836322+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:58.130702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013471519999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":1274,"num_read_kb":10604,"num_write":428,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"170'983","reported_seq":6790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837827+0000","last_change":"2024-09-14T22:55:51.397026+0000","last_active":"2024-09-14T22:56:09.837827+0000","last_peered":"2024-09-14T22:56:09.837827+0000","last_clean":"2024-09-14T22:56:09.837827+0000","last_became_active":"2024-09-14T22:55:51.143935+0000","last_became_peered":"2024-09-14T22:55:51.143935+0000","last_unstale":"2024-09-14T22:56:09.837827+0000","last_undegraded":"2024-09-14T22:56:09.837827+0000","last_fullsized":"2024-09-14T22:56:09.837827+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:40:42.066436+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034758323000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":8863,"num_read_kb":12560,"num_write":1145,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":1377,"num_read":260757,"num_read_kb":1066701,"num_write":19012,"num_write_kb":702896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":141629825,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},"log_size":18820,"ondisk_log_size":18820,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56204,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315636,"statfs":{"total":96636764160,"available":96579211264,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29252,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342588,"statfs":{"total":96636764160,"available":96606810112,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51460,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320380,"statfs":{"total":96636764160,"available":96584069120,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":169,"seq":725849473030,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35772,"kb_used_data":2216,"kb_used_omap":47,"kb_used_meta":33488,"kb_avail":94336068,"statfs":{"total":96636764160,"available":96600133632,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49036,"internal_metadata":34291828},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2265088,"data_stored":1107477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41732,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330108,"statfs":{"total":96636764160,"available":96594030592,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29264,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342576,"statfs":{"total":96636764160,"available":96606797824,"internally_reserved":0,"allocated":2285568,"data_stored":1113560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56208,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315632,"statfs":{"total":96636764160,"available":96579207168,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:56:12.743 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:56:13.022 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:13.022 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:56:13.033 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-14T22:56:12.819536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1385,"num_object_clones":0,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":1377,"num_read":260873,"num_read_kb":1066801,"num_write":19173,"num_write_kb":705314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":141629863,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18940,"ondisk_log_size":18940,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333300,"kb_used_data":18932,"kb_used_omap":58,"kb_used_meta":314117,"kb_avail":754641420,"statfs":{"total":773094113280,"available":772752814080,"internally_reserved":0,"allocated":19386368,"data_stored":10070853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60192,"internal_metadata":321656032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.999850"},"pg_stats":[{"pgid":"3.1f","version":"162'679","reported_seq":8427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836344+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836344+0000","last_peered":"2024-09-14T22:56:09.836344+0000","last_clean":"2024-09-14T22:56:09.836344+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:56:09.836344+0000","last_undegraded":"2024-09-14T22:56:09.836344+0000","last_fullsized":"2024-09-14T22:56:09.836344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:24:55.707066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001547748,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":9382,"num_read_kb":31310,"num_write":714,"num_write_kb":7692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"160'529","reported_seq":8618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313112+0000","last_change":"2024-09-14T22:55:36.055650+0000","last_active":"2024-09-14T22:56:10.313112+0000","last_peered":"2024-09-14T22:56:10.313112+0000","last_clean":"2024-09-14T22:56:10.313112+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:56:10.313112+0000","last_undegraded":"2024-09-14T22:56:10.313112+0000","last_fullsized":"2024-09-14T22:56:10.313112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:03:40.697937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012941300000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8409,"num_read_kb":20232,"num_write":463,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"170'521","reported_seq":9286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313783+0000","last_change":"2024-09-14T22:55:51.254209+0000","last_active":"2024-09-14T22:56:10.313783+0000","last_peered":"2024-09-14T22:56:10.313783+0000","last_clean":"2024-09-14T22:56:10.313783+0000","last_became_active":"2024-09-14T22:55:51.142689+0000","last_became_peered":"2024-09-14T22:55:51.142689+0000","last_unstale":"2024-09-14T22:56:10.313783+0000","last_undegraded":"2024-09-14T22:56:10.313783+0000","last_fullsized":"2024-09-14T22:56:10.313783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:02.520144+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.057378967000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9221,"num_read_kb":38644,"num_write":441,"num_write_kb":52016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"170'649","reported_seq":3618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313632+0000","last_change":"2024-09-14T22:55:51.339722+0000","last_active":"2024-09-14T22:56:10.313632+0000","last_peered":"2024-09-14T22:56:10.313632+0000","last_clean":"2024-09-14T22:56:10.313632+0000","last_became_active":"2024-09-14T22:55:51.144955+0000","last_became_peered":"2024-09-14T22:55:51.144955+0000","last_unstale":"2024-09-14T22:56:10.313632+0000","last_undegraded":"2024-09-14T22:56:10.313632+0000","last_fullsized":"2024-09-14T22:56:10.313632+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:33:11.938624+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.037609783000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":4493,"num_read_kb":12183,"num_write":718,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"170'340","reported_seq":5375,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836439+0000","last_change":"2024-09-14T22:55:51.202792+0000","last_active":"2024-09-14T22:56:09.836439+0000","last_peered":"2024-09-14T22:56:09.836439+0000","last_clean":"2024-09-14T22:56:09.836439+0000","last_became_active":"2024-09-14T22:55:51.137980+0000","last_became_peered":"2024-09-14T22:55:51.137980+0000","last_unstale":"2024-09-14T22:56:09.836439+0000","last_undegraded":"2024-09-14T22:56:09.836439+0000","last_fullsized":"2024-09-14T22:56:09.836439+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:12.182377+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034914117000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":4616,"num_read_kb":54005,"num_write":312,"num_write_kb":18747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"155'691","reported_seq":14554,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314213+0000","last_change":"2024-09-14T22:55:53.767098+0000","last_active":"2024-09-14T22:56:10.314213+0000","last_peered":"2024-09-14T22:56:10.314213+0000","last_clean":"2024-09-14T22:56:10.314213+0000","last_became_active":"2024-09-14T22:55:53.765594+0000","last_became_peered":"2024-09-14T22:55:53.765594+0000","last_unstale":"2024-09-14T22:56:10.314213+0000","last_undegraded":"2024-09-14T22:56:10.314213+0000","last_fullsized":"2024-09-14T22:56:10.314213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:56.211691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071064000000000001,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":16209,"num_read_kb":53499,"num_write":537,"num_write_kb":29099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":18306298,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"160'485","reported_seq":16175,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835713+0000","last_change":"2024-09-14T22:55:58.780115+0000","last_active":"2024-09-14T22:56:09.835713+0000","last_peered":"2024-09-14T22:56:09.835713+0000","last_clean":"2024-09-14T22:56:09.835713+0000","last_became_active":"2024-09-14T22:55:58.778624+0000","last_became_peered":"2024-09-14T22:55:58.778624+0000","last_unstale":"2024-09-14T22:56:09.835713+0000","last_undegraded":"2024-09-14T22:56:09.835713+0000","last_fullsized":"2024-09-14T22:56:09.835713+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:43.711519+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059748170000000003,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":15953,"num_read_kb":92561,"num_write":414,"num_write_kb":38470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'675","reported_seq":2449,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312849+0000","last_change":"2024-09-14T22:55:36.056150+0000","last_active":"2024-09-14T22:56:10.312849+0000","last_peered":"2024-09-14T22:56:10.312849+0000","last_clean":"2024-09-14T22:56:10.312849+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:56:10.312849+0000","last_undegraded":"2024-09-14T22:56:10.312849+0000","last_fullsized":"2024-09-14T22:56:10.312849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:58.129978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001629828,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":2959,"num_read_kb":22740,"num_write":861,"num_write_kb":10481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"161'652","reported_seq":6924,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836420+0000","last_change":"2024-09-14T22:55:56.769135+0000","last_active":"2024-09-14T22:56:09.836420+0000","last_peered":"2024-09-14T22:56:09.836420+0000","last_clean":"2024-09-14T22:56:09.836420+0000","last_became_active":"2024-09-14T22:55:56.768221+0000","last_became_peered":"2024-09-14T22:55:56.768221+0000","last_unstale":"2024-09-14T22:56:09.836420+0000","last_undegraded":"2024-09-14T22:56:09.836420+0000","last_fullsized":"2024-09-14T22:56:09.836420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:42.862372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001569408,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":7874,"num_read_kb":34799,"num_write":817,"num_write_kb":27727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"161'433","reported_seq":9778,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313725+0000","last_change":"2024-09-14T22:55:36.057943+0000","last_active":"2024-09-14T22:56:10.313725+0000","last_peered":"2024-09-14T22:56:10.313725+0000","last_clean":"2024-09-14T22:56:10.313725+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:56:10.313725+0000","last_undegraded":"2024-09-14T22:56:10.313725+0000","last_fullsized":"2024-09-14T22:56:10.313725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:01.697087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011312640000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10352,"num_read_kb":46008,"num_write":347,"num_write_kb":32677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"161'421","reported_seq":9422,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837884+0000","last_change":"2024-09-14T22:55:36.135035+0000","last_active":"2024-09-14T22:56:09.837884+0000","last_peered":"2024-09-14T22:56:09.837884+0000","last_clean":"2024-09-14T22:56:09.837884+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:56:09.837884+0000","last_undegraded":"2024-09-14T22:56:09.837884+0000","last_fullsized":"2024-09-14T22:56:09.837884+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:30.224048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086910600000000002,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10004,"num_read_kb":43389,"num_write":402,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"170'601","reported_seq":10915,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314273+0000","last_change":"2024-09-14T22:55:51.257123+0000","last_active":"2024-09-14T22:56:10.314273+0000","last_peered":"2024-09-14T22:56:10.314273+0000","last_clean":"2024-09-14T22:56:10.314273+0000","last_became_active":"2024-09-14T22:55:51.138619+0000","last_became_peered":"2024-09-14T22:55:51.138619+0000","last_unstale":"2024-09-14T22:56:10.314273+0000","last_undegraded":"2024-09-14T22:56:10.314273+0000","last_fullsized":"2024-09-14T22:56:10.314273+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:13.451285+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.074291557999999994,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11380,"num_read_kb":66402,"num_write":623,"num_write_kb":37521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313940+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:56:10.313940+0000","last_peered":"2024-09-14T22:56:10.313940+0000","last_clean":"2024-09-14T22:56:10.313940+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:56:10.313940+0000","last_undegraded":"2024-09-14T22:56:10.313940+0000","last_fullsized":"2024-09-14T22:56:10.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"162'728","reported_seq":8066,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836307+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836307+0000","last_peered":"2024-09-14T22:56:09.836307+0000","last_clean":"2024-09-14T22:56:09.836307+0000","last_became_active":"2024-09-14T22:55:32.040243+0000","last_became_peered":"2024-09-14T22:55:32.040243+0000","last_unstale":"2024-09-14T22:56:09.836307+0000","last_undegraded":"2024-09-14T22:56:09.836307+0000","last_fullsized":"2024-09-14T22:56:09.836307+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:28:24.384824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012928849999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":9421,"num_read_kb":42199,"num_write":751,"num_write_kb":24667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":310,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313779+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:56:10.313779+0000","last_peered":"2024-09-14T22:56:10.313779+0000","last_clean":"2024-09-14T22:56:10.313779+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:56:10.313779+0000","last_undegraded":"2024-09-14T22:56:10.313779+0000","last_fullsized":"2024-09-14T22:56:10.313779+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"156'335","reported_seq":2125,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837835+0000","last_change":"2024-09-14T22:55:36.134620+0000","last_active":"2024-09-14T22:56:09.837835+0000","last_peered":"2024-09-14T22:56:09.837835+0000","last_clean":"2024-09-14T22:56:09.837835+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:56:09.837835+0000","last_undegraded":"2024-09-14T22:56:09.837835+0000","last_fullsized":"2024-09-14T22:56:09.837835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:44.383091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079363000000000003,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":1234,"num_read_kb":15244,"num_write":279,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837945+0000","last_change":"2024-09-14T22:55:39.451745+0000","last_active":"2024-09-14T22:56:09.837945+0000","last_peered":"2024-09-14T22:56:09.837945+0000","last_clean":"2024-09-14T22:56:09.837945+0000","last_became_active":"2024-09-14T22:55:39.451421+0000","last_became_peered":"2024-09-14T22:55:39.451421+0000","last_unstale":"2024-09-14T22:56:09.837945+0000","last_undegraded":"2024-09-14T22:56:09.837945+0000","last_fullsized":"2024-09-14T22:56:09.837945+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:48.524842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"161'844","reported_seq":14297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835861+0000","last_change":"2024-09-14T22:55:58.778184+0000","last_active":"2024-09-14T22:56:09.835861+0000","last_peered":"2024-09-14T22:56:09.835861+0000","last_clean":"2024-09-14T22:56:09.835861+0000","last_became_active":"2024-09-14T22:55:58.776278+0000","last_became_peered":"2024-09-14T22:55:58.776278+0000","last_unstale":"2024-09-14T22:56:09.835861+0000","last_undegraded":"2024-09-14T22:56:09.835861+0000","last_fullsized":"2024-09-14T22:56:09.835861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:19:29.875793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992360000000002,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":15582,"num_read_kb":40098,"num_write":1081,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":15786495,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835949+0000","last_change":"2024-09-14T22:55:58.776581+0000","last_active":"2024-09-14T22:56:09.835949+0000","last_peered":"2024-09-14T22:56:09.835949+0000","last_clean":"2024-09-14T22:56:09.835949+0000","last_became_active":"2024-09-14T22:55:58.776284+0000","last_became_peered":"2024-09-14T22:55:58.776284+0000","last_unstale":"2024-09-14T22:56:09.835949+0000","last_undegraded":"2024-09-14T22:56:09.835949+0000","last_fullsized":"2024-09-14T22:56:09.835949+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:35.328965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"161'434","reported_seq":15790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313853+0000","last_change":"2024-09-14T22:55:36.057245+0000","last_active":"2024-09-14T22:56:10.313853+0000","last_peered":"2024-09-14T22:56:10.313853+0000","last_clean":"2024-09-14T22:56:10.313853+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:56:10.313853+0000","last_undegraded":"2024-09-14T22:56:10.313853+0000","last_fullsized":"2024-09-14T22:56:10.313853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:39:45.146222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055810799999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14507,"num_read_kb":48865,"num_write":357,"num_write_kb":19125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"162'1001","reported_seq":6842,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312863+0000","last_change":"2024-09-14T22:55:36.109343+0000","last_active":"2024-09-14T22:56:10.312863+0000","last_peered":"2024-09-14T22:56:10.312863+0000","last_clean":"2024-09-14T22:56:10.312863+0000","last_became_active":"2024-09-14T22:54:47.747387+0000","last_became_peered":"2024-09-14T22:54:47.747387+0000","last_unstale":"2024-09-14T22:56:10.312863+0000","last_undegraded":"2024-09-14T22:56:10.312863+0000","last_fullsized":"2024-09-14T22:56:10.312863+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:27.684254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356700000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":9071,"num_read_kb":57942,"num_write":1041,"num_write_kb":41152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":29933802,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313688+0000","last_change":"2024-09-14T22:55:32.043590+0000","last_active":"2024-09-14T22:56:10.313688+0000","last_peered":"2024-09-14T22:56:10.313688+0000","last_clean":"2024-09-14T22:56:10.313688+0000","last_became_active":"2024-09-14T22:55:32.040648+0000","last_became_peered":"2024-09-14T22:55:32.040648+0000","last_unstale":"2024-09-14T22:56:10.313688+0000","last_undegraded":"2024-09-14T22:56:10.313688+0000","last_fullsized":"2024-09-14T22:56:10.313688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:45:24.150360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"161'370","reported_seq":1908,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313752+0000","last_change":"2024-09-14T22:55:36.062326+0000","last_active":"2024-09-14T22:56:10.313752+0000","last_peered":"2024-09-14T22:56:10.313752+0000","last_clean":"2024-09-14T22:56:10.313752+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:56:10.313752+0000","last_undegraded":"2024-09-14T22:56:10.313752+0000","last_fullsized":"2024-09-14T22:56:10.313752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:43:31.326081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140828,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":1390,"num_read_kb":9489,"num_write":336,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":363,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312875+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:56:10.312875+0000","last_peered":"2024-09-14T22:56:10.312875+0000","last_clean":"2024-09-14T22:56:10.312875+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:56:10.312875+0000","last_undegraded":"2024-09-14T22:56:10.312875+0000","last_fullsized":"2024-09-14T22:56:10.312875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"170'632","reported_seq":9392,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836259+0000","last_change":"2024-09-14T22:55:51.310755+0000","last_active":"2024-09-14T22:56:09.836259+0000","last_peered":"2024-09-14T22:56:09.836259+0000","last_clean":"2024-09-14T22:56:09.836259+0000","last_became_active":"2024-09-14T22:55:51.139441+0000","last_became_peered":"2024-09-14T22:55:51.139441+0000","last_unstale":"2024-09-14T22:56:09.836259+0000","last_undegraded":"2024-09-14T22:56:09.836259+0000","last_fullsized":"2024-09-14T22:56:09.836259+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:50.139115+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.026769095999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":10199,"num_read_kb":21422,"num_write":735,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837668+0000","last_change":"2024-09-14T22:55:32.040027+0000","last_active":"2024-09-14T22:56:09.837668+0000","last_peered":"2024-09-14T22:56:09.837668+0000","last_clean":"2024-09-14T22:56:09.837668+0000","last_became_active":"2024-09-14T22:55:32.039728+0000","last_became_peered":"2024-09-14T22:55:32.039728+0000","last_unstale":"2024-09-14T22:56:09.837668+0000","last_undegraded":"2024-09-14T22:56:09.837668+0000","last_fullsized":"2024-09-14T22:56:09.837668+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:36.095183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"170'487","reported_seq":6942,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314372+0000","last_change":"2024-09-14T22:55:51.267041+0000","last_active":"2024-09-14T22:56:10.314372+0000","last_peered":"2024-09-14T22:56:10.314372+0000","last_clean":"2024-09-14T22:56:10.314372+0000","last_became_active":"2024-09-14T22:55:51.141016+0000","last_became_peered":"2024-09-14T22:55:51.141016+0000","last_unstale":"2024-09-14T22:56:10.314372+0000","last_undegraded":"2024-09-14T22:56:10.314372+0000","last_fullsized":"2024-09-14T22:56:10.314372+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:00.580008+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.063591116000000003,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":6223,"num_read_kb":32196,"num_write":426,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313602+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:56:10.313602+0000","last_peered":"2024-09-14T22:56:10.313602+0000","last_clean":"2024-09-14T22:56:10.313602+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:56:10.313602+0000","last_undegraded":"2024-09-14T22:56:10.313602+0000","last_fullsized":"2024-09-14T22:56:10.313602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":503,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837732+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:56:09.837732+0000","last_peered":"2024-09-14T22:56:09.837732+0000","last_clean":"2024-09-14T22:56:09.837732+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:56:09.837732+0000","last_undegraded":"2024-09-14T22:56:09.837732+0000","last_fullsized":"2024-09-14T22:56:09.837732+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'940","reported_seq":10279,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312917+0000","last_change":"2024-09-14T22:55:36.056644+0000","last_active":"2024-09-14T22:56:10.312917+0000","last_peered":"2024-09-14T22:56:10.312917+0000","last_clean":"2024-09-14T22:56:10.312917+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:56:10.312917+0000","last_undegraded":"2024-09-14T22:56:10.312917+0000","last_fullsized":"2024-09-14T22:56:10.312917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:54.515748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019587290000000002,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":14513,"num_read_kb":36136,"num_write":948,"num_write_kb":13116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'531","reported_seq":2277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314277+0000","last_change":"2024-09-14T22:55:51.198762+0000","last_active":"2024-09-14T22:56:10.314277+0000","last_peered":"2024-09-14T22:56:10.314277+0000","last_clean":"2024-09-14T22:56:10.314277+0000","last_became_active":"2024-09-14T22:55:51.135840+0000","last_became_peered":"2024-09-14T22:55:51.135840+0000","last_unstale":"2024-09-14T22:56:10.314277+0000","last_undegraded":"2024-09-14T22:56:10.314277+0000","last_fullsized":"2024-09-14T22:56:10.314277+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:51.299112+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.046042974,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2706,"num_read_kb":9423,"num_write":453,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"170'531","reported_seq":2506,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313894+0000","last_change":"2024-09-14T22:55:51.296897+0000","last_active":"2024-09-14T22:56:10.313894+0000","last_peered":"2024-09-14T22:56:10.313894+0000","last_clean":"2024-09-14T22:56:10.313894+0000","last_became_active":"2024-09-14T22:55:51.143629+0000","last_became_peered":"2024-09-14T22:55:51.143629+0000","last_unstale":"2024-09-14T22:56:10.313894+0000","last_undegraded":"2024-09-14T22:56:10.313894+0000","last_fullsized":"2024-09-14T22:56:10.313894+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:51.928400+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.037412189999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":36,"num_read":3234,"num_read_kb":8551,"num_write":522,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'689","reported_seq":4144,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313011+0000","last_change":"2024-09-14T22:55:36.056675+0000","last_active":"2024-09-14T22:56:10.313011+0000","last_peered":"2024-09-14T22:56:10.313011+0000","last_clean":"2024-09-14T22:56:10.313011+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:56:10.313011+0000","last_undegraded":"2024-09-14T22:56:10.313011+0000","last_fullsized":"2024-09-14T22:56:10.313011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:48:33.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001452467,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5954,"num_read_kb":20565,"num_write":618,"num_write_kb":24613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'577","reported_seq":5751,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313749+0000","last_change":"2024-09-14T22:55:36.057396+0000","last_active":"2024-09-14T22:56:10.313749+0000","last_peered":"2024-09-14T22:56:10.313749+0000","last_clean":"2024-09-14T22:56:10.313749+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:56:10.313749+0000","last_undegraded":"2024-09-14T22:56:10.313749+0000","last_fullsized":"2024-09-14T22:56:10.313749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:58.321946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000939155,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":6243,"num_read_kb":22048,"num_write":553,"num_write_kb":16971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"162'626","reported_seq":7317,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837797+0000","last_change":"2024-09-14T22:55:36.135696+0000","last_active":"2024-09-14T22:56:09.837797+0000","last_peered":"2024-09-14T22:56:09.837797+0000","last_clean":"2024-09-14T22:56:09.837797+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:56:09.837797+0000","last_undegraded":"2024-09-14T22:56:09.837797+0000","last_fullsized":"2024-09-14T22:56:09.837797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:57.110923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014953080000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":8194,"num_read_kb":36336,"num_write":770,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"170'515","reported_seq":7151,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.838025+0000","last_change":"2024-09-14T22:55:51.374445+0000","last_active":"2024-09-14T22:56:09.838025+0000","last_peered":"2024-09-14T22:56:09.838025+0000","last_clean":"2024-09-14T22:56:09.838025+0000","last_became_active":"2024-09-14T22:55:51.141871+0000","last_became_peered":"2024-09-14T22:55:51.141871+0000","last_unstale":"2024-09-14T22:56:09.838025+0000","last_undegraded":"2024-09-14T22:56:09.838025+0000","last_fullsized":"2024-09-14T22:56:09.838025+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:57:25.784874+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.053472498,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":7727,"num_read_kb":40328,"num_write":448,"num_write_kb":21333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"170'348","reported_seq":6502,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837958+0000","last_change":"2024-09-14T22:55:51.268238+0000","last_active":"2024-09-14T22:56:09.837958+0000","last_peered":"2024-09-14T22:56:09.837958+0000","last_clean":"2024-09-14T22:56:09.837958+0000","last_became_active":"2024-09-14T22:55:51.141931+0000","last_became_peered":"2024-09-14T22:55:51.141931+0000","last_unstale":"2024-09-14T22:56:09.837958+0000","last_undegraded":"2024-09-14T22:56:09.837958+0000","last_fullsized":"2024-09-14T22:56:09.837958+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:22.125200+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.047107007999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5750,"num_read_kb":21575,"num_write":279,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'550","reported_seq":10161,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313774+0000","last_change":"2024-09-14T22:55:36.062330+0000","last_active":"2024-09-14T22:56:10.313774+0000","last_peered":"2024-09-14T22:56:10.313774+0000","last_clean":"2024-09-14T22:56:10.313774+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:56:10.313774+0000","last_undegraded":"2024-09-14T22:56:10.313774+0000","last_fullsized":"2024-09-14T22:56:10.313774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:02.042762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128364,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":11616,"num_read_kb":30610,"num_write":481,"num_write_kb":42134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"170'613","reported_seq":5711,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837789+0000","last_change":"2024-09-14T22:55:51.388404+0000","last_active":"2024-09-14T22:56:09.837789+0000","last_peered":"2024-09-14T22:56:09.837789+0000","last_clean":"2024-09-14T22:56:09.837789+0000","last_became_active":"2024-09-14T22:55:51.142779+0000","last_became_peered":"2024-09-14T22:55:51.142779+0000","last_unstale":"2024-09-14T22:56:09.837789+0000","last_undegraded":"2024-09-14T22:56:09.837789+0000","last_fullsized":"2024-09-14T22:56:09.837789+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:34.034047+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.041728379000000003,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":6204,"num_read_kb":34738,"num_write":712,"num_write_kb":24203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"161'410","reported_seq":1626,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836322+0000","last_change":"2024-09-14T22:55:56.769064+0000","last_active":"2024-09-14T22:56:09.836322+0000","last_peered":"2024-09-14T22:56:09.836322+0000","last_clean":"2024-09-14T22:56:09.836322+0000","last_became_active":"2024-09-14T22:55:56.767773+0000","last_became_peered":"2024-09-14T22:55:56.767773+0000","last_unstale":"2024-09-14T22:56:09.836322+0000","last_undegraded":"2024-09-14T22:56:09.836322+0000","last_fullsized":"2024-09-14T22:56:09.836322+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:58.130702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013471519999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":1274,"num_read_kb":10604,"num_write":428,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"170'983","reported_seq":6790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837827+0000","last_change":"2024-09-14T22:55:51.397026+0000","last_active":"2024-09-14T22:56:09.837827+0000","last_peered":"2024-09-14T22:56:09.837827+0000","last_clean":"2024-09-14T22:56:09.837827+0000","last_became_active":"2024-09-14T22:55:51.143935+0000","last_became_peered":"2024-09-14T22:55:51.143935+0000","last_unstale":"2024-09-14T22:56:09.837827+0000","last_undegraded":"2024-09-14T22:56:09.837827+0000","last_fullsized":"2024-09-14T22:56:09.837827+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:40:42.066436+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034758323000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":8863,"num_read_kb":12560,"num_write":1145,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":1377,"num_read":260757,"num_read_kb":1066701,"num_write":19012,"num_write_kb":702896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":141629825,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},"log_size":18820,"ondisk_log_size":18820,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56204,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315636,"statfs":{"total":96636764160,"available":96579211264,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29252,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342588,"statfs":{"total":96636764160,"available":96606810112,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51460,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320380,"statfs":{"total":96636764160,"available":96584069120,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":169,"seq":725849473030,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35772,"kb_used_data":2216,"kb_used_omap":47,"kb_used_meta":33488,"kb_avail":94336068,"statfs":{"total":96636764160,"available":96600133632,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49036,"internal_metadata":34291828},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2265088,"data_stored":1107477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41732,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330108,"statfs":{"total":96636764160,"available":96594030592,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29264,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342576,"statfs":{"total":96636764160,"available":96606797824,"internally_reserved":0,"allocated":2285568,"data_stored":1113560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56208,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315632,"statfs":{"total":96636764160,"available":96579207168,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:56:13.035 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T22:56:13.035 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:56:13.312 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:13.312 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:56:13.324 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-14T22:56:12.819536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1385,"num_object_clones":0,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":1377,"num_read":260873,"num_read_kb":1066801,"num_write":19173,"num_write_kb":705314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":141629863,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18940,"ondisk_log_size":18940,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333300,"kb_used_data":18932,"kb_used_omap":58,"kb_used_meta":314117,"kb_avail":754641420,"statfs":{"total":773094113280,"available":772752814080,"internally_reserved":0,"allocated":19386368,"data_stored":10070853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60192,"internal_metadata":321656032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.999850"},"pg_stats":[{"pgid":"3.1f","version":"162'679","reported_seq":8427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836344+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836344+0000","last_peered":"2024-09-14T22:56:09.836344+0000","last_clean":"2024-09-14T22:56:09.836344+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:56:09.836344+0000","last_undegraded":"2024-09-14T22:56:09.836344+0000","last_fullsized":"2024-09-14T22:56:09.836344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:24:55.707066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001547748,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":9382,"num_read_kb":31310,"num_write":714,"num_write_kb":7692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"160'529","reported_seq":8618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313112+0000","last_change":"2024-09-14T22:55:36.055650+0000","last_active":"2024-09-14T22:56:10.313112+0000","last_peered":"2024-09-14T22:56:10.313112+0000","last_clean":"2024-09-14T22:56:10.313112+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:56:10.313112+0000","last_undegraded":"2024-09-14T22:56:10.313112+0000","last_fullsized":"2024-09-14T22:56:10.313112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:03:40.697937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012941300000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8409,"num_read_kb":20232,"num_write":463,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"170'521","reported_seq":9286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313783+0000","last_change":"2024-09-14T22:55:51.254209+0000","last_active":"2024-09-14T22:56:10.313783+0000","last_peered":"2024-09-14T22:56:10.313783+0000","last_clean":"2024-09-14T22:56:10.313783+0000","last_became_active":"2024-09-14T22:55:51.142689+0000","last_became_peered":"2024-09-14T22:55:51.142689+0000","last_unstale":"2024-09-14T22:56:10.313783+0000","last_undegraded":"2024-09-14T22:56:10.313783+0000","last_fullsized":"2024-09-14T22:56:10.313783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:02.520144+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.057378967000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9221,"num_read_kb":38644,"num_write":441,"num_write_kb":52016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"170'649","reported_seq":3618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313632+0000","last_change":"2024-09-14T22:55:51.339722+0000","last_active":"2024-09-14T22:56:10.313632+0000","last_peered":"2024-09-14T22:56:10.313632+0000","last_clean":"2024-09-14T22:56:10.313632+0000","last_became_active":"2024-09-14T22:55:51.144955+0000","last_became_peered":"2024-09-14T22:55:51.144955+0000","last_unstale":"2024-09-14T22:56:10.313632+0000","last_undegraded":"2024-09-14T22:56:10.313632+0000","last_fullsized":"2024-09-14T22:56:10.313632+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:33:11.938624+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.037609783000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":4493,"num_read_kb":12183,"num_write":718,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"170'340","reported_seq":5375,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836439+0000","last_change":"2024-09-14T22:55:51.202792+0000","last_active":"2024-09-14T22:56:09.836439+0000","last_peered":"2024-09-14T22:56:09.836439+0000","last_clean":"2024-09-14T22:56:09.836439+0000","last_became_active":"2024-09-14T22:55:51.137980+0000","last_became_peered":"2024-09-14T22:55:51.137980+0000","last_unstale":"2024-09-14T22:56:09.836439+0000","last_undegraded":"2024-09-14T22:56:09.836439+0000","last_fullsized":"2024-09-14T22:56:09.836439+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:12.182377+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034914117000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":4616,"num_read_kb":54005,"num_write":312,"num_write_kb":18747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"155'691","reported_seq":14554,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314213+0000","last_change":"2024-09-14T22:55:53.767098+0000","last_active":"2024-09-14T22:56:10.314213+0000","last_peered":"2024-09-14T22:56:10.314213+0000","last_clean":"2024-09-14T22:56:10.314213+0000","last_became_active":"2024-09-14T22:55:53.765594+0000","last_became_peered":"2024-09-14T22:55:53.765594+0000","last_unstale":"2024-09-14T22:56:10.314213+0000","last_undegraded":"2024-09-14T22:56:10.314213+0000","last_fullsized":"2024-09-14T22:56:10.314213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:56.211691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071064000000000001,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":16209,"num_read_kb":53499,"num_write":537,"num_write_kb":29099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":18306298,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"160'485","reported_seq":16175,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835713+0000","last_change":"2024-09-14T22:55:58.780115+0000","last_active":"2024-09-14T22:56:09.835713+0000","last_peered":"2024-09-14T22:56:09.835713+0000","last_clean":"2024-09-14T22:56:09.835713+0000","last_became_active":"2024-09-14T22:55:58.778624+0000","last_became_peered":"2024-09-14T22:55:58.778624+0000","last_unstale":"2024-09-14T22:56:09.835713+0000","last_undegraded":"2024-09-14T22:56:09.835713+0000","last_fullsized":"2024-09-14T22:56:09.835713+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:43.711519+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059748170000000003,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":15953,"num_read_kb":92561,"num_write":414,"num_write_kb":38470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'675","reported_seq":2449,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312849+0000","last_change":"2024-09-14T22:55:36.056150+0000","last_active":"2024-09-14T22:56:10.312849+0000","last_peered":"2024-09-14T22:56:10.312849+0000","last_clean":"2024-09-14T22:56:10.312849+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:56:10.312849+0000","last_undegraded":"2024-09-14T22:56:10.312849+0000","last_fullsized":"2024-09-14T22:56:10.312849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:58.129978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001629828,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":2959,"num_read_kb":22740,"num_write":861,"num_write_kb":10481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"161'652","reported_seq":6924,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836420+0000","last_change":"2024-09-14T22:55:56.769135+0000","last_active":"2024-09-14T22:56:09.836420+0000","last_peered":"2024-09-14T22:56:09.836420+0000","last_clean":"2024-09-14T22:56:09.836420+0000","last_became_active":"2024-09-14T22:55:56.768221+0000","last_became_peered":"2024-09-14T22:55:56.768221+0000","last_unstale":"2024-09-14T22:56:09.836420+0000","last_undegraded":"2024-09-14T22:56:09.836420+0000","last_fullsized":"2024-09-14T22:56:09.836420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:42.862372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001569408,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":7874,"num_read_kb":34799,"num_write":817,"num_write_kb":27727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"161'433","reported_seq":9778,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313725+0000","last_change":"2024-09-14T22:55:36.057943+0000","last_active":"2024-09-14T22:56:10.313725+0000","last_peered":"2024-09-14T22:56:10.313725+0000","last_clean":"2024-09-14T22:56:10.313725+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:56:10.313725+0000","last_undegraded":"2024-09-14T22:56:10.313725+0000","last_fullsized":"2024-09-14T22:56:10.313725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:01.697087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011312640000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10352,"num_read_kb":46008,"num_write":347,"num_write_kb":32677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"161'421","reported_seq":9422,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837884+0000","last_change":"2024-09-14T22:55:36.135035+0000","last_active":"2024-09-14T22:56:09.837884+0000","last_peered":"2024-09-14T22:56:09.837884+0000","last_clean":"2024-09-14T22:56:09.837884+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:56:09.837884+0000","last_undegraded":"2024-09-14T22:56:09.837884+0000","last_fullsized":"2024-09-14T22:56:09.837884+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:30.224048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086910600000000002,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10004,"num_read_kb":43389,"num_write":402,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"170'601","reported_seq":10915,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314273+0000","last_change":"2024-09-14T22:55:51.257123+0000","last_active":"2024-09-14T22:56:10.314273+0000","last_peered":"2024-09-14T22:56:10.314273+0000","last_clean":"2024-09-14T22:56:10.314273+0000","last_became_active":"2024-09-14T22:55:51.138619+0000","last_became_peered":"2024-09-14T22:55:51.138619+0000","last_unstale":"2024-09-14T22:56:10.314273+0000","last_undegraded":"2024-09-14T22:56:10.314273+0000","last_fullsized":"2024-09-14T22:56:10.314273+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:13.451285+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.074291557999999994,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11380,"num_read_kb":66402,"num_write":623,"num_write_kb":37521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313940+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:56:10.313940+0000","last_peered":"2024-09-14T22:56:10.313940+0000","last_clean":"2024-09-14T22:56:10.313940+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:56:10.313940+0000","last_undegraded":"2024-09-14T22:56:10.313940+0000","last_fullsized":"2024-09-14T22:56:10.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"162'728","reported_seq":8066,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836307+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836307+0000","last_peered":"2024-09-14T22:56:09.836307+0000","last_clean":"2024-09-14T22:56:09.836307+0000","last_became_active":"2024-09-14T22:55:32.040243+0000","last_became_peered":"2024-09-14T22:55:32.040243+0000","last_unstale":"2024-09-14T22:56:09.836307+0000","last_undegraded":"2024-09-14T22:56:09.836307+0000","last_fullsized":"2024-09-14T22:56:09.836307+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:28:24.384824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012928849999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":9421,"num_read_kb":42199,"num_write":751,"num_write_kb":24667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":310,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313779+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:56:10.313779+0000","last_peered":"2024-09-14T22:56:10.313779+0000","last_clean":"2024-09-14T22:56:10.313779+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:56:10.313779+0000","last_undegraded":"2024-09-14T22:56:10.313779+0000","last_fullsized":"2024-09-14T22:56:10.313779+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"156'335","reported_seq":2125,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837835+0000","last_change":"2024-09-14T22:55:36.134620+0000","last_active":"2024-09-14T22:56:09.837835+0000","last_peered":"2024-09-14T22:56:09.837835+0000","last_clean":"2024-09-14T22:56:09.837835+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:56:09.837835+0000","last_undegraded":"2024-09-14T22:56:09.837835+0000","last_fullsized":"2024-09-14T22:56:09.837835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:44.383091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079363000000000003,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":1234,"num_read_kb":15244,"num_write":279,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837945+0000","last_change":"2024-09-14T22:55:39.451745+0000","last_active":"2024-09-14T22:56:09.837945+0000","last_peered":"2024-09-14T22:56:09.837945+0000","last_clean":"2024-09-14T22:56:09.837945+0000","last_became_active":"2024-09-14T22:55:39.451421+0000","last_became_peered":"2024-09-14T22:55:39.451421+0000","last_unstale":"2024-09-14T22:56:09.837945+0000","last_undegraded":"2024-09-14T22:56:09.837945+0000","last_fullsized":"2024-09-14T22:56:09.837945+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:48.524842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"161'844","reported_seq":14297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835861+0000","last_change":"2024-09-14T22:55:58.778184+0000","last_active":"2024-09-14T22:56:09.835861+0000","last_peered":"2024-09-14T22:56:09.835861+0000","last_clean":"2024-09-14T22:56:09.835861+0000","last_became_active":"2024-09-14T22:55:58.776278+0000","last_became_peered":"2024-09-14T22:55:58.776278+0000","last_unstale":"2024-09-14T22:56:09.835861+0000","last_undegraded":"2024-09-14T22:56:09.835861+0000","last_fullsized":"2024-09-14T22:56:09.835861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:19:29.875793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992360000000002,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":15582,"num_read_kb":40098,"num_write":1081,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":15786495,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835949+0000","last_change":"2024-09-14T22:55:58.776581+0000","last_active":"2024-09-14T22:56:09.835949+0000","last_peered":"2024-09-14T22:56:09.835949+0000","last_clean":"2024-09-14T22:56:09.835949+0000","last_became_active":"2024-09-14T22:55:58.776284+0000","last_became_peered":"2024-09-14T22:55:58.776284+0000","last_unstale":"2024-09-14T22:56:09.835949+0000","last_undegraded":"2024-09-14T22:56:09.835949+0000","last_fullsized":"2024-09-14T22:56:09.835949+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:35.328965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"161'434","reported_seq":15790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313853+0000","last_change":"2024-09-14T22:55:36.057245+0000","last_active":"2024-09-14T22:56:10.313853+0000","last_peered":"2024-09-14T22:56:10.313853+0000","last_clean":"2024-09-14T22:56:10.313853+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:56:10.313853+0000","last_undegraded":"2024-09-14T22:56:10.313853+0000","last_fullsized":"2024-09-14T22:56:10.313853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:39:45.146222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055810799999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14507,"num_read_kb":48865,"num_write":357,"num_write_kb":19125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"162'1001","reported_seq":6842,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312863+0000","last_change":"2024-09-14T22:55:36.109343+0000","last_active":"2024-09-14T22:56:10.312863+0000","last_peered":"2024-09-14T22:56:10.312863+0000","last_clean":"2024-09-14T22:56:10.312863+0000","last_became_active":"2024-09-14T22:54:47.747387+0000","last_became_peered":"2024-09-14T22:54:47.747387+0000","last_unstale":"2024-09-14T22:56:10.312863+0000","last_undegraded":"2024-09-14T22:56:10.312863+0000","last_fullsized":"2024-09-14T22:56:10.312863+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:27.684254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356700000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":9071,"num_read_kb":57942,"num_write":1041,"num_write_kb":41152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":29933802,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313688+0000","last_change":"2024-09-14T22:55:32.043590+0000","last_active":"2024-09-14T22:56:10.313688+0000","last_peered":"2024-09-14T22:56:10.313688+0000","last_clean":"2024-09-14T22:56:10.313688+0000","last_became_active":"2024-09-14T22:55:32.040648+0000","last_became_peered":"2024-09-14T22:55:32.040648+0000","last_unstale":"2024-09-14T22:56:10.313688+0000","last_undegraded":"2024-09-14T22:56:10.313688+0000","last_fullsized":"2024-09-14T22:56:10.313688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:45:24.150360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"161'370","reported_seq":1908,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313752+0000","last_change":"2024-09-14T22:55:36.062326+0000","last_active":"2024-09-14T22:56:10.313752+0000","last_peered":"2024-09-14T22:56:10.313752+0000","last_clean":"2024-09-14T22:56:10.313752+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:56:10.313752+0000","last_undegraded":"2024-09-14T22:56:10.313752+0000","last_fullsized":"2024-09-14T22:56:10.313752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:43:31.326081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140828,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":1390,"num_read_kb":9489,"num_write":336,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":363,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312875+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:56:10.312875+0000","last_peered":"2024-09-14T22:56:10.312875+0000","last_clean":"2024-09-14T22:56:10.312875+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:56:10.312875+0000","last_undegraded":"2024-09-14T22:56:10.312875+0000","last_fullsized":"2024-09-14T22:56:10.312875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"170'632","reported_seq":9392,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836259+0000","last_change":"2024-09-14T22:55:51.310755+0000","last_active":"2024-09-14T22:56:09.836259+0000","last_peered":"2024-09-14T22:56:09.836259+0000","last_clean":"2024-09-14T22:56:09.836259+0000","last_became_active":"2024-09-14T22:55:51.139441+0000","last_became_peered":"2024-09-14T22:55:51.139441+0000","last_unstale":"2024-09-14T22:56:09.836259+0000","last_undegraded":"2024-09-14T22:56:09.836259+0000","last_fullsized":"2024-09-14T22:56:09.836259+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:50.139115+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.026769095999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":10199,"num_read_kb":21422,"num_write":735,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837668+0000","last_change":"2024-09-14T22:55:32.040027+0000","last_active":"2024-09-14T22:56:09.837668+0000","last_peered":"2024-09-14T22:56:09.837668+0000","last_clean":"2024-09-14T22:56:09.837668+0000","last_became_active":"2024-09-14T22:55:32.039728+0000","last_became_peered":"2024-09-14T22:55:32.039728+0000","last_unstale":"2024-09-14T22:56:09.837668+0000","last_undegraded":"2024-09-14T22:56:09.837668+0000","last_fullsized":"2024-09-14T22:56:09.837668+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:36.095183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"170'487","reported_seq":6942,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314372+0000","last_change":"2024-09-14T22:55:51.267041+0000","last_active":"2024-09-14T22:56:10.314372+0000","last_peered":"2024-09-14T22:56:10.314372+0000","last_clean":"2024-09-14T22:56:10.314372+0000","last_became_active":"2024-09-14T22:55:51.141016+0000","last_became_peered":"2024-09-14T22:55:51.141016+0000","last_unstale":"2024-09-14T22:56:10.314372+0000","last_undegraded":"2024-09-14T22:56:10.314372+0000","last_fullsized":"2024-09-14T22:56:10.314372+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:00.580008+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.063591116000000003,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":6223,"num_read_kb":32196,"num_write":426,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313602+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:56:10.313602+0000","last_peered":"2024-09-14T22:56:10.313602+0000","last_clean":"2024-09-14T22:56:10.313602+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:56:10.313602+0000","last_undegraded":"2024-09-14T22:56:10.313602+0000","last_fullsized":"2024-09-14T22:56:10.313602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":503,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837732+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:56:09.837732+0000","last_peered":"2024-09-14T22:56:09.837732+0000","last_clean":"2024-09-14T22:56:09.837732+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:56:09.837732+0000","last_undegraded":"2024-09-14T22:56:09.837732+0000","last_fullsized":"2024-09-14T22:56:09.837732+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'940","reported_seq":10279,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312917+0000","last_change":"2024-09-14T22:55:36.056644+0000","last_active":"2024-09-14T22:56:10.312917+0000","last_peered":"2024-09-14T22:56:10.312917+0000","last_clean":"2024-09-14T22:56:10.312917+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:56:10.312917+0000","last_undegraded":"2024-09-14T22:56:10.312917+0000","last_fullsized":"2024-09-14T22:56:10.312917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:54.515748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019587290000000002,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":14513,"num_read_kb":36136,"num_write":948,"num_write_kb":13116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'531","reported_seq":2277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314277+0000","last_change":"2024-09-14T22:55:51.198762+0000","last_active":"2024-09-14T22:56:10.314277+0000","last_peered":"2024-09-14T22:56:10.314277+0000","last_clean":"2024-09-14T22:56:10.314277+0000","last_became_active":"2024-09-14T22:55:51.135840+0000","last_became_peered":"2024-09-14T22:55:51.135840+0000","last_unstale":"2024-09-14T22:56:10.314277+0000","last_undegraded":"2024-09-14T22:56:10.314277+0000","last_fullsized":"2024-09-14T22:56:10.314277+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:51.299112+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.046042974,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2706,"num_read_kb":9423,"num_write":453,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"170'531","reported_seq":2506,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313894+0000","last_change":"2024-09-14T22:55:51.296897+0000","last_active":"2024-09-14T22:56:10.313894+0000","last_peered":"2024-09-14T22:56:10.313894+0000","last_clean":"2024-09-14T22:56:10.313894+0000","last_became_active":"2024-09-14T22:55:51.143629+0000","last_became_peered":"2024-09-14T22:55:51.143629+0000","last_unstale":"2024-09-14T22:56:10.313894+0000","last_undegraded":"2024-09-14T22:56:10.313894+0000","last_fullsized":"2024-09-14T22:56:10.313894+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:51.928400+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.037412189999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":36,"num_read":3234,"num_read_kb":8551,"num_write":522,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'689","reported_seq":4144,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313011+0000","last_change":"2024-09-14T22:55:36.056675+0000","last_active":"2024-09-14T22:56:10.313011+0000","last_peered":"2024-09-14T22:56:10.313011+0000","last_clean":"2024-09-14T22:56:10.313011+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:56:10.313011+0000","last_undegraded":"2024-09-14T22:56:10.313011+0000","last_fullsized":"2024-09-14T22:56:10.313011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:48:33.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001452467,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5954,"num_read_kb":20565,"num_write":618,"num_write_kb":24613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'577","reported_seq":5751,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313749+0000","last_change":"2024-09-14T22:55:36.057396+0000","last_active":"2024-09-14T22:56:10.313749+0000","last_peered":"2024-09-14T22:56:10.313749+0000","last_clean":"2024-09-14T22:56:10.313749+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:56:10.313749+0000","last_undegraded":"2024-09-14T22:56:10.313749+0000","last_fullsized":"2024-09-14T22:56:10.313749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:58.321946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000939155,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":6243,"num_read_kb":22048,"num_write":553,"num_write_kb":16971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"162'626","reported_seq":7317,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837797+0000","last_change":"2024-09-14T22:55:36.135696+0000","last_active":"2024-09-14T22:56:09.837797+0000","last_peered":"2024-09-14T22:56:09.837797+0000","last_clean":"2024-09-14T22:56:09.837797+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:56:09.837797+0000","last_undegraded":"2024-09-14T22:56:09.837797+0000","last_fullsized":"2024-09-14T22:56:09.837797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:57.110923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014953080000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":8194,"num_read_kb":36336,"num_write":770,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"170'515","reported_seq":7151,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.838025+0000","last_change":"2024-09-14T22:55:51.374445+0000","last_active":"2024-09-14T22:56:09.838025+0000","last_peered":"2024-09-14T22:56:09.838025+0000","last_clean":"2024-09-14T22:56:09.838025+0000","last_became_active":"2024-09-14T22:55:51.141871+0000","last_became_peered":"2024-09-14T22:55:51.141871+0000","last_unstale":"2024-09-14T22:56:09.838025+0000","last_undegraded":"2024-09-14T22:56:09.838025+0000","last_fullsized":"2024-09-14T22:56:09.838025+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:57:25.784874+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.053472498,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":7727,"num_read_kb":40328,"num_write":448,"num_write_kb":21333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"170'348","reported_seq":6502,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837958+0000","last_change":"2024-09-14T22:55:51.268238+0000","last_active":"2024-09-14T22:56:09.837958+0000","last_peered":"2024-09-14T22:56:09.837958+0000","last_clean":"2024-09-14T22:56:09.837958+0000","last_became_active":"2024-09-14T22:55:51.141931+0000","last_became_peered":"2024-09-14T22:55:51.141931+0000","last_unstale":"2024-09-14T22:56:09.837958+0000","last_undegraded":"2024-09-14T22:56:09.837958+0000","last_fullsized":"2024-09-14T22:56:09.837958+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:22.125200+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.047107007999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5750,"num_read_kb":21575,"num_write":279,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'550","reported_seq":10161,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313774+0000","last_change":"2024-09-14T22:55:36.062330+0000","last_active":"2024-09-14T22:56:10.313774+0000","last_peered":"2024-09-14T22:56:10.313774+0000","last_clean":"2024-09-14T22:56:10.313774+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:56:10.313774+0000","last_undegraded":"2024-09-14T22:56:10.313774+0000","last_fullsized":"2024-09-14T22:56:10.313774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:02.042762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128364,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":11616,"num_read_kb":30610,"num_write":481,"num_write_kb":42134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"170'613","reported_seq":5711,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837789+0000","last_change":"2024-09-14T22:55:51.388404+0000","last_active":"2024-09-14T22:56:09.837789+0000","last_peered":"2024-09-14T22:56:09.837789+0000","last_clean":"2024-09-14T22:56:09.837789+0000","last_became_active":"2024-09-14T22:55:51.142779+0000","last_became_peered":"2024-09-14T22:55:51.142779+0000","last_unstale":"2024-09-14T22:56:09.837789+0000","last_undegraded":"2024-09-14T22:56:09.837789+0000","last_fullsized":"2024-09-14T22:56:09.837789+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:34.034047+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.041728379000000003,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":6204,"num_read_kb":34738,"num_write":712,"num_write_kb":24203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"161'410","reported_seq":1626,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836322+0000","last_change":"2024-09-14T22:55:56.769064+0000","last_active":"2024-09-14T22:56:09.836322+0000","last_peered":"2024-09-14T22:56:09.836322+0000","last_clean":"2024-09-14T22:56:09.836322+0000","last_became_active":"2024-09-14T22:55:56.767773+0000","last_became_peered":"2024-09-14T22:55:56.767773+0000","last_unstale":"2024-09-14T22:56:09.836322+0000","last_undegraded":"2024-09-14T22:56:09.836322+0000","last_fullsized":"2024-09-14T22:56:09.836322+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:58.130702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013471519999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":1274,"num_read_kb":10604,"num_write":428,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"170'983","reported_seq":6790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837827+0000","last_change":"2024-09-14T22:55:51.397026+0000","last_active":"2024-09-14T22:56:09.837827+0000","last_peered":"2024-09-14T22:56:09.837827+0000","last_clean":"2024-09-14T22:56:09.837827+0000","last_became_active":"2024-09-14T22:55:51.143935+0000","last_became_peered":"2024-09-14T22:55:51.143935+0000","last_unstale":"2024-09-14T22:56:09.837827+0000","last_undegraded":"2024-09-14T22:56:09.837827+0000","last_fullsized":"2024-09-14T22:56:09.837827+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:40:42.066436+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034758323000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":8863,"num_read_kb":12560,"num_write":1145,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":1377,"num_read":260757,"num_read_kb":1066701,"num_write":19012,"num_write_kb":702896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":141629825,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},"log_size":18820,"ondisk_log_size":18820,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56204,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315636,"statfs":{"total":96636764160,"available":96579211264,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29252,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342588,"statfs":{"total":96636764160,"available":96606810112,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51460,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320380,"statfs":{"total":96636764160,"available":96584069120,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":169,"seq":725849473030,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35772,"kb_used_data":2216,"kb_used_omap":47,"kb_used_meta":33488,"kb_avail":94336068,"statfs":{"total":96636764160,"available":96600133632,"internally_reserved":0,"allocated":2269184,"data_stored":1107496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49036,"internal_metadata":34291828},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2265088,"data_stored":1107477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41732,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330108,"statfs":{"total":96636764160,"available":96594030592,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29264,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342576,"statfs":{"total":96636764160,"available":96606797824,"internally_reserved":0,"allocated":2285568,"data_stored":1113560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56208,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315632,"statfs":{"total":96636764160,"available":96579207168,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:56:13.326 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T22:56:13.602 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:13.602 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":189,"fsid":"9a42a40e-8d25-4801-9835-f823a294bdbe","created":"2024-09-14T22:52:37.944451+0000","modified":"2024-09-14T22:56:09.832649+0000","last_up_change":"2024-09-14T22:55:50.117726+0000","last_in_change":"2024-09-14T22:55:31.022383+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T22:52:50.813186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T22:52:55.912547+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":2.630000114440918,"raw_score_stable":2.630000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T22:53:05.643169+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":162,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45f61e20-37e6-4a13-b005-1f135f5e5a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6801","nonce":2488220599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6803","nonce":2488220599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6807","nonce":2488220599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2488220599},{"type":"v1","addr":"172.21.15.76:6805","nonce":2488220599}]},"public_addr":"172.21.15.76:6801/2488220599","cluster_addr":"172.21.15.76:6803/2488220599","heartbeat_back_addr":"172.21.15.76:6807/2488220599","heartbeat_front_addr":"172.21.15.76:6805/2488220599","state":["exists","up"]},{"osd":1,"uuid":"bc72773d-098f-4ac5-89b5-13dae87b65b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6809","nonce":522829438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6811","nonce":522829438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6815","nonce":522829438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":522829438},{"type":"v1","addr":"172.21.15.76:6813","nonce":522829438}]},"public_addr":"172.21.15.76:6809/522829438","cluster_addr":"172.21.15.76:6811/522829438","heartbeat_back_addr":"172.21.15.76:6815/522829438","heartbeat_front_addr":"172.21.15.76:6813/522829438","state":["exists","up"]},{"osd":2,"uuid":"2c2a50b0-e6ab-4593-9a26-badf1cc3dd34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6825","nonce":3214755929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6827","nonce":3214755929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6831","nonce":3214755929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":3214755929},{"type":"v1","addr":"172.21.15.76:6829","nonce":3214755929}]},"public_addr":"172.21.15.76:6825/3214755929","cluster_addr":"172.21.15.76:6827/3214755929","heartbeat_back_addr":"172.21.15.76:6831/3214755929","heartbeat_front_addr":"172.21.15.76:6829/3214755929","state":["exists","up"]},{"osd":3,"uuid":"40d20d5f-5509-4a9f-8b13-c9e8da7339bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6817","nonce":2699236014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6819","nonce":2699236014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6823","nonce":2699236014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2699236014},{"type":"v1","addr":"172.21.15.76:6821","nonce":2699236014}]},"public_addr":"172.21.15.76:6817/2699236014","cluster_addr":"172.21.15.76:6819/2699236014","heartbeat_back_addr":"172.21.15.76:6823/2699236014","heartbeat_front_addr":"172.21.15.76:6821/2699236014","state":["exists","up"]},{"osd":4,"uuid":"78bf5206-fbf7-4668-a27f-9eb867edc4e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":123,"up_from":169,"up_thru":172,"down_at":124,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6825","nonce":3093265760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6827","nonce":3093265760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6831","nonce":3093265760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":3093265760},{"type":"v1","addr":"172.21.15.114:6829","nonce":3093265760}]},"public_addr":"172.21.15.114:6825/3093265760","cluster_addr":"172.21.15.114:6827/3093265760","heartbeat_back_addr":"172.21.15.114:6831/3093265760","heartbeat_front_addr":"172.21.15.114:6829/3093265760","state":["exists","up"]},{"osd":5,"uuid":"72838af1-36f2-4cf6-a5ee-6daf5d4ea70a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6801","nonce":3469663218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6803","nonce":3469663218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6807","nonce":3469663218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":3469663218},{"type":"v1","addr":"172.21.15.114:6805","nonce":3469663218}]},"public_addr":"172.21.15.114:6801/3469663218","cluster_addr":"172.21.15.114:6803/3469663218","heartbeat_back_addr":"172.21.15.114:6807/3469663218","heartbeat_front_addr":"172.21.15.114:6805/3469663218","state":["exists","up"]},{"osd":6,"uuid":"11be725a-e81f-4b8a-8a68-acb54e67b443","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6817","nonce":2551471767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6819","nonce":2551471767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6823","nonce":2551471767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2551471767},{"type":"v1","addr":"172.21.15.114:6821","nonce":2551471767}]},"public_addr":"172.21.15.114:6817/2551471767","cluster_addr":"172.21.15.114:6819/2551471767","heartbeat_back_addr":"172.21.15.114:6823/2551471767","heartbeat_front_addr":"172.21.15.114:6821/2551471767","state":["exists","up"]},{"osd":7,"uuid":"94103011-716e-495f-aeb3-759071ce6f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6809","nonce":941014947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6811","nonce":941014947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6815","nonce":941014947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":941014947},{"type":"v1","addr":"172.21.15.114:6813","nonce":941014947}]},"public_addr":"172.21.15.114:6809/941014947","cluster_addr":"172.21.15.114:6811/941014947","heartbeat_back_addr":"172.21.15.114:6815/941014947","heartbeat_front_addr":"172.21.15.114:6813/941014947","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.133153+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.415439+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.880448+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.879405+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:47.079602+0000","dead_epoch":124},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.493580+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.693553+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T22:52:46.442044+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[7,6]},{"pgid":"2.6","osds":[5,2]},{"pgid":"3.0","osds":[6,7]},{"pgid":"3.2","osds":[4,5]},{"pgid":"3.7","osds":[7,6]},{"pgid":"3.8","osds":[3,6]}],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":2}]},{"pgid":"3.12","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T22:56:14.616 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T22:56:14.616 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:56:14.754 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:56:14.754 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:56:14.754 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:56:14.765 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T22:56:15.044 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T22:56:15.056 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T22:56:15.057 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:56:15.180 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:56:15.180 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:56:15.180 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:56:15.191 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T22:56:15.468 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T22:56:15.480 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T22:56:15.481 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:56:15.604 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:56:15.614 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T22:56:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T22:56:15.908 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T22:56:15.908 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:56:16.030 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:56:16.031 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:56:16.031 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:56:16.041 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T22:56:16.325 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T22:56:16.338 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T22:56:16.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-14T22:56:16.462 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:56:16.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T22:56:16.750 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T22:56:16.763 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T22:56:16.763 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:56:16.887 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:56:16.898 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T22:56:17.204 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T22:56:17.217 INFO:tasks.ceph:Scrubbing osd.6 2024-09-14T22:56:17.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:56:17.345 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:56:17.356 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-14T22:56:17.631 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 6 to deep-scrub 2024-09-14T22:56:17.644 INFO:tasks.ceph:Scrubbing osd.7 2024-09-14T22:56:17.644 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-14T22:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-14T22:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T22:56:17.777 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-14T22:56:17.788 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-14T22:56:18.088 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 7 to deep-scrub 2024-09-14T22:56:18.103 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:56:18.383 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:18.383 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:56:18.394 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-14T22:56:16.820339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1385,"num_object_clones":0,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":1377,"num_read":260873,"num_read_kb":1066801,"num_write":19173,"num_write_kb":705314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":141629863,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18940,"ondisk_log_size":18940,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":333324,"kb_used_data":18968,"kb_used_omap":58,"kb_used_meta":314117,"kb_avail":754641396,"statfs":{"total":773094113280,"available":772752789504,"internally_reserved":0,"allocated":19423232,"data_stored":10088988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60192,"internal_metadata":321656032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001765"},"pg_stats":[{"pgid":"3.1f","version":"162'679","reported_seq":8427,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836344+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836344+0000","last_peered":"2024-09-14T22:56:09.836344+0000","last_clean":"2024-09-14T22:56:09.836344+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:56:09.836344+0000","last_undegraded":"2024-09-14T22:56:09.836344+0000","last_fullsized":"2024-09-14T22:56:09.836344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:24:55.707066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001547748,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":9382,"num_read_kb":31310,"num_write":714,"num_write_kb":7692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"160'529","reported_seq":8618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313112+0000","last_change":"2024-09-14T22:55:36.055650+0000","last_active":"2024-09-14T22:56:10.313112+0000","last_peered":"2024-09-14T22:56:10.313112+0000","last_clean":"2024-09-14T22:56:10.313112+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:56:10.313112+0000","last_undegraded":"2024-09-14T22:56:10.313112+0000","last_fullsized":"2024-09-14T22:56:10.313112+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:03:40.697937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012941300000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8409,"num_read_kb":20232,"num_write":463,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"170'521","reported_seq":9286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313783+0000","last_change":"2024-09-14T22:55:51.254209+0000","last_active":"2024-09-14T22:56:10.313783+0000","last_peered":"2024-09-14T22:56:10.313783+0000","last_clean":"2024-09-14T22:56:10.313783+0000","last_became_active":"2024-09-14T22:55:51.142689+0000","last_became_peered":"2024-09-14T22:55:51.142689+0000","last_unstale":"2024-09-14T22:56:10.313783+0000","last_undegraded":"2024-09-14T22:56:10.313783+0000","last_fullsized":"2024-09-14T22:56:10.313783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:02.520144+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.057378967000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9221,"num_read_kb":38644,"num_write":441,"num_write_kb":52016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"170'649","reported_seq":3618,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313632+0000","last_change":"2024-09-14T22:55:51.339722+0000","last_active":"2024-09-14T22:56:10.313632+0000","last_peered":"2024-09-14T22:56:10.313632+0000","last_clean":"2024-09-14T22:56:10.313632+0000","last_became_active":"2024-09-14T22:55:51.144955+0000","last_became_peered":"2024-09-14T22:55:51.144955+0000","last_unstale":"2024-09-14T22:56:10.313632+0000","last_undegraded":"2024-09-14T22:56:10.313632+0000","last_fullsized":"2024-09-14T22:56:10.313632+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:33:11.938624+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.037609783000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":4493,"num_read_kb":12183,"num_write":718,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"170'340","reported_seq":5375,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836439+0000","last_change":"2024-09-14T22:55:51.202792+0000","last_active":"2024-09-14T22:56:09.836439+0000","last_peered":"2024-09-14T22:56:09.836439+0000","last_clean":"2024-09-14T22:56:09.836439+0000","last_became_active":"2024-09-14T22:55:51.137980+0000","last_became_peered":"2024-09-14T22:55:51.137980+0000","last_unstale":"2024-09-14T22:56:09.836439+0000","last_undegraded":"2024-09-14T22:56:09.836439+0000","last_fullsized":"2024-09-14T22:56:09.836439+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:37:12.182377+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034914117000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":4616,"num_read_kb":54005,"num_write":312,"num_write_kb":18747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"155'691","reported_seq":14554,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314213+0000","last_change":"2024-09-14T22:55:53.767098+0000","last_active":"2024-09-14T22:56:10.314213+0000","last_peered":"2024-09-14T22:56:10.314213+0000","last_clean":"2024-09-14T22:56:10.314213+0000","last_became_active":"2024-09-14T22:55:53.765594+0000","last_became_peered":"2024-09-14T22:55:53.765594+0000","last_unstale":"2024-09-14T22:56:10.314213+0000","last_undegraded":"2024-09-14T22:56:10.314213+0000","last_fullsized":"2024-09-14T22:56:10.314213+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:48:56.211691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071064000000000001,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":16209,"num_read_kb":53499,"num_write":537,"num_write_kb":29099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":18306298,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"160'485","reported_seq":16175,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835713+0000","last_change":"2024-09-14T22:55:58.780115+0000","last_active":"2024-09-14T22:56:09.835713+0000","last_peered":"2024-09-14T22:56:09.835713+0000","last_clean":"2024-09-14T22:56:09.835713+0000","last_became_active":"2024-09-14T22:55:58.778624+0000","last_became_peered":"2024-09-14T22:55:58.778624+0000","last_unstale":"2024-09-14T22:56:09.835713+0000","last_undegraded":"2024-09-14T22:56:09.835713+0000","last_fullsized":"2024-09-14T22:56:09.835713+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:48:43.711519+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059748170000000003,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":15953,"num_read_kb":92561,"num_write":414,"num_write_kb":38470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"161'675","reported_seq":2449,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312849+0000","last_change":"2024-09-14T22:55:36.056150+0000","last_active":"2024-09-14T22:56:10.312849+0000","last_peered":"2024-09-14T22:56:10.312849+0000","last_clean":"2024-09-14T22:56:10.312849+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:56:10.312849+0000","last_undegraded":"2024-09-14T22:56:10.312849+0000","last_fullsized":"2024-09-14T22:56:10.312849+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:21:58.129978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001629828,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":2959,"num_read_kb":22740,"num_write":861,"num_write_kb":10481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"161'652","reported_seq":6924,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836420+0000","last_change":"2024-09-14T22:55:56.769135+0000","last_active":"2024-09-14T22:56:09.836420+0000","last_peered":"2024-09-14T22:56:09.836420+0000","last_clean":"2024-09-14T22:56:09.836420+0000","last_became_active":"2024-09-14T22:55:56.768221+0000","last_became_peered":"2024-09-14T22:55:56.768221+0000","last_unstale":"2024-09-14T22:56:09.836420+0000","last_undegraded":"2024-09-14T22:56:09.836420+0000","last_fullsized":"2024-09-14T22:56:09.836420+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:42.862372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001569408,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":7874,"num_read_kb":34799,"num_write":817,"num_write_kb":27727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"161'433","reported_seq":9778,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313725+0000","last_change":"2024-09-14T22:55:36.057943+0000","last_active":"2024-09-14T22:56:10.313725+0000","last_peered":"2024-09-14T22:56:10.313725+0000","last_clean":"2024-09-14T22:56:10.313725+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:56:10.313725+0000","last_undegraded":"2024-09-14T22:56:10.313725+0000","last_fullsized":"2024-09-14T22:56:10.313725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:02:01.697087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011312640000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10352,"num_read_kb":46008,"num_write":347,"num_write_kb":32677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"161'421","reported_seq":9422,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837884+0000","last_change":"2024-09-14T22:55:36.135035+0000","last_active":"2024-09-14T22:56:09.837884+0000","last_peered":"2024-09-14T22:56:09.837884+0000","last_clean":"2024-09-14T22:56:09.837884+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:56:09.837884+0000","last_undegraded":"2024-09-14T22:56:09.837884+0000","last_fullsized":"2024-09-14T22:56:09.837884+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:30.224048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086910600000000002,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10004,"num_read_kb":43389,"num_write":402,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"170'601","reported_seq":10915,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314273+0000","last_change":"2024-09-14T22:55:51.257123+0000","last_active":"2024-09-14T22:56:10.314273+0000","last_peered":"2024-09-14T22:56:10.314273+0000","last_clean":"2024-09-14T22:56:10.314273+0000","last_became_active":"2024-09-14T22:55:51.138619+0000","last_became_peered":"2024-09-14T22:55:51.138619+0000","last_unstale":"2024-09-14T22:56:10.314273+0000","last_undegraded":"2024-09-14T22:56:10.314273+0000","last_fullsized":"2024-09-14T22:56:10.314273+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:08:13.451285+0000","scrub_duration":0,"objects_trimmed":32,"snaptrim_duration":0.074291557999999994,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11380,"num_read_kb":66402,"num_write":623,"num_write_kb":37521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313940+0000","last_change":"2024-09-14T22:52:59.823204+0000","last_active":"2024-09-14T22:56:10.313940+0000","last_peered":"2024-09-14T22:56:10.313940+0000","last_clean":"2024-09-14T22:56:10.313940+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:56:10.313940+0000","last_undegraded":"2024-09-14T22:56:10.313940+0000","last_fullsized":"2024-09-14T22:56:10.313940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:39:47.429183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"162'728","reported_seq":8066,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836307+0000","last_change":"2024-09-14T22:55:36.056231+0000","last_active":"2024-09-14T22:56:09.836307+0000","last_peered":"2024-09-14T22:56:09.836307+0000","last_clean":"2024-09-14T22:56:09.836307+0000","last_became_active":"2024-09-14T22:55:32.040243+0000","last_became_peered":"2024-09-14T22:55:32.040243+0000","last_unstale":"2024-09-14T22:56:09.836307+0000","last_undegraded":"2024-09-14T22:56:09.836307+0000","last_fullsized":"2024-09-14T22:56:09.836307+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:28:24.384824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012928849999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":9421,"num_read_kb":42199,"num_write":751,"num_write_kb":24667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":310,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313779+0000","last_change":"2024-09-14T22:53:21.944201+0000","last_active":"2024-09-14T22:56:10.313779+0000","last_peered":"2024-09-14T22:56:10.313779+0000","last_clean":"2024-09-14T22:56:10.313779+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:56:10.313779+0000","last_undegraded":"2024-09-14T22:56:10.313779+0000","last_fullsized":"2024-09-14T22:56:10.313779+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:45:49.185238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"156'335","reported_seq":2126,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837884+0000","last_change":"2024-09-14T22:55:36.134620+0000","last_active":"2024-09-14T22:56:09.837884+0000","last_peered":"2024-09-14T22:56:09.837884+0000","last_clean":"2024-09-14T22:56:09.837884+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:56:09.837884+0000","last_undegraded":"2024-09-14T22:56:09.837884+0000","last_fullsized":"2024-09-14T22:56:09.837884+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:01:44.383091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079363000000000003,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":1234,"num_read_kb":15244,"num_write":279,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837945+0000","last_change":"2024-09-14T22:55:39.451745+0000","last_active":"2024-09-14T22:56:09.837945+0000","last_peered":"2024-09-14T22:56:09.837945+0000","last_clean":"2024-09-14T22:56:09.837945+0000","last_became_active":"2024-09-14T22:55:39.451421+0000","last_became_peered":"2024-09-14T22:55:39.451421+0000","last_unstale":"2024-09-14T22:56:09.837945+0000","last_undegraded":"2024-09-14T22:56:09.837945+0000","last_fullsized":"2024-09-14T22:56:09.837945+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:55:48.524842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"161'844","reported_seq":14297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835861+0000","last_change":"2024-09-14T22:55:58.778184+0000","last_active":"2024-09-14T22:56:09.835861+0000","last_peered":"2024-09-14T22:56:09.835861+0000","last_clean":"2024-09-14T22:56:09.835861+0000","last_became_active":"2024-09-14T22:55:58.776278+0000","last_became_peered":"2024-09-14T22:55:58.776278+0000","last_unstale":"2024-09-14T22:56:09.835861+0000","last_undegraded":"2024-09-14T22:56:09.835861+0000","last_fullsized":"2024-09-14T22:56:09.835861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:19:29.875793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992360000000002,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":15582,"num_read_kb":40098,"num_write":1081,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":15786495,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.835949+0000","last_change":"2024-09-14T22:55:58.776581+0000","last_active":"2024-09-14T22:56:09.835949+0000","last_peered":"2024-09-14T22:56:09.835949+0000","last_clean":"2024-09-14T22:56:09.835949+0000","last_became_active":"2024-09-14T22:55:58.776284+0000","last_became_peered":"2024-09-14T22:55:58.776284+0000","last_unstale":"2024-09-14T22:56:09.835949+0000","last_undegraded":"2024-09-14T22:56:09.835949+0000","last_fullsized":"2024-09-14T22:56:09.835949+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:51:35.328965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"161'434","reported_seq":15790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313853+0000","last_change":"2024-09-14T22:55:36.057245+0000","last_active":"2024-09-14T22:56:10.313853+0000","last_peered":"2024-09-14T22:56:10.313853+0000","last_clean":"2024-09-14T22:56:10.313853+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:56:10.313853+0000","last_undegraded":"2024-09-14T22:56:10.313853+0000","last_fullsized":"2024-09-14T22:56:10.313853+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:39:45.146222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055810799999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14507,"num_read_kb":48865,"num_write":357,"num_write_kb":19125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"162'1001","reported_seq":6842,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312863+0000","last_change":"2024-09-14T22:55:36.109343+0000","last_active":"2024-09-14T22:56:10.312863+0000","last_peered":"2024-09-14T22:56:10.312863+0000","last_clean":"2024-09-14T22:56:10.312863+0000","last_became_active":"2024-09-14T22:54:47.747387+0000","last_became_peered":"2024-09-14T22:54:47.747387+0000","last_unstale":"2024-09-14T22:56:10.312863+0000","last_undegraded":"2024-09-14T22:56:10.312863+0000","last_fullsized":"2024-09-14T22:56:10.312863+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:36:27.684254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356700000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":9071,"num_read_kb":57942,"num_write":1041,"num_write_kb":41152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":29933802,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313688+0000","last_change":"2024-09-14T22:55:32.043590+0000","last_active":"2024-09-14T22:56:10.313688+0000","last_peered":"2024-09-14T22:56:10.313688+0000","last_clean":"2024-09-14T22:56:10.313688+0000","last_became_active":"2024-09-14T22:55:32.040648+0000","last_became_peered":"2024-09-14T22:55:32.040648+0000","last_unstale":"2024-09-14T22:56:10.313688+0000","last_undegraded":"2024-09-14T22:56:10.313688+0000","last_fullsized":"2024-09-14T22:56:10.313688+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:45:24.150360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"161'370","reported_seq":1908,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313752+0000","last_change":"2024-09-14T22:55:36.062326+0000","last_active":"2024-09-14T22:56:10.313752+0000","last_peered":"2024-09-14T22:56:10.313752+0000","last_clean":"2024-09-14T22:56:10.313752+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:56:10.313752+0000","last_undegraded":"2024-09-14T22:56:10.313752+0000","last_fullsized":"2024-09-14T22:56:10.313752+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:43:31.326081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140828,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":1390,"num_read_kb":9489,"num_write":336,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":363,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312875+0000","last_change":"2024-09-14T22:54:25.161205+0000","last_active":"2024-09-14T22:56:10.312875+0000","last_peered":"2024-09-14T22:56:10.312875+0000","last_clean":"2024-09-14T22:56:10.312875+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:56:10.312875+0000","last_undegraded":"2024-09-14T22:56:10.312875+0000","last_fullsized":"2024-09-14T22:56:10.312875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:02:39.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"170'632","reported_seq":9392,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836259+0000","last_change":"2024-09-14T22:55:51.310755+0000","last_active":"2024-09-14T22:56:09.836259+0000","last_peered":"2024-09-14T22:56:09.836259+0000","last_clean":"2024-09-14T22:56:09.836259+0000","last_became_active":"2024-09-14T22:55:51.139441+0000","last_became_peered":"2024-09-14T22:55:51.139441+0000","last_unstale":"2024-09-14T22:56:09.836259+0000","last_undegraded":"2024-09-14T22:56:09.836259+0000","last_fullsized":"2024-09-14T22:56:09.836259+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:10:50.139115+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.026769095999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":10199,"num_read_kb":21422,"num_write":735,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837668+0000","last_change":"2024-09-14T22:55:32.040027+0000","last_active":"2024-09-14T22:56:09.837668+0000","last_peered":"2024-09-14T22:56:09.837668+0000","last_clean":"2024-09-14T22:56:09.837668+0000","last_became_active":"2024-09-14T22:55:32.039728+0000","last_became_peered":"2024-09-14T22:55:32.039728+0000","last_unstale":"2024-09-14T22:56:09.837668+0000","last_undegraded":"2024-09-14T22:56:09.837668+0000","last_fullsized":"2024-09-14T22:56:09.837668+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:09:36.095183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"170'487","reported_seq":6942,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314372+0000","last_change":"2024-09-14T22:55:51.267041+0000","last_active":"2024-09-14T22:56:10.314372+0000","last_peered":"2024-09-14T22:56:10.314372+0000","last_clean":"2024-09-14T22:56:10.314372+0000","last_became_active":"2024-09-14T22:55:51.141016+0000","last_became_peered":"2024-09-14T22:55:51.141016+0000","last_unstale":"2024-09-14T22:56:10.314372+0000","last_undegraded":"2024-09-14T22:56:10.314372+0000","last_fullsized":"2024-09-14T22:56:10.314372+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:24:00.580008+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.063591116000000003,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":6223,"num_read_kb":32196,"num_write":426,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313602+0000","last_change":"2024-09-14T22:52:59.536124+0000","last_active":"2024-09-14T22:56:10.313602+0000","last_peered":"2024-09-14T22:56:10.313602+0000","last_clean":"2024-09-14T22:56:10.313602+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:56:10.313602+0000","last_undegraded":"2024-09-14T22:56:10.313602+0000","last_fullsized":"2024-09-14T22:56:10.313602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:56.511969+0000","last_clean_scrub_stamp":"2024-09-14T22:52:56.511969+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:31.656217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":503,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837732+0000","last_change":"2024-09-14T22:52:52.654758+0000","last_active":"2024-09-14T22:56:09.837732+0000","last_peered":"2024-09-14T22:56:09.837732+0000","last_clean":"2024-09-14T22:56:09.837732+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:56:09.837732+0000","last_undegraded":"2024-09-14T22:56:09.837732+0000","last_fullsized":"2024-09-14T22:56:09.837732+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:52:51.486958+0000","last_clean_scrub_stamp":"2024-09-14T22:52:51.486958+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:22:28.887394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"162'940","reported_seq":10279,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.312917+0000","last_change":"2024-09-14T22:55:36.056644+0000","last_active":"2024-09-14T22:56:10.312917+0000","last_peered":"2024-09-14T22:56:10.312917+0000","last_clean":"2024-09-14T22:56:10.312917+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:56:10.312917+0000","last_undegraded":"2024-09-14T22:56:10.312917+0000","last_fullsized":"2024-09-14T22:56:10.312917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:39:54.515748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019587290000000002,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":14513,"num_read_kb":36136,"num_write":948,"num_write_kb":13116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"170'531","reported_seq":2277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.314277+0000","last_change":"2024-09-14T22:55:51.198762+0000","last_active":"2024-09-14T22:56:10.314277+0000","last_peered":"2024-09-14T22:56:10.314277+0000","last_clean":"2024-09-14T22:56:10.314277+0000","last_became_active":"2024-09-14T22:55:51.135840+0000","last_became_peered":"2024-09-14T22:55:51.135840+0000","last_unstale":"2024-09-14T22:56:10.314277+0000","last_undegraded":"2024-09-14T22:56:10.314277+0000","last_fullsized":"2024-09-14T22:56:10.314277+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:57:51.299112+0000","scrub_duration":0,"objects_trimmed":26,"snaptrim_duration":0.046042974,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2706,"num_read_kb":9423,"num_write":453,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"170'531","reported_seq":2506,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313894+0000","last_change":"2024-09-14T22:55:51.296897+0000","last_active":"2024-09-14T22:56:10.313894+0000","last_peered":"2024-09-14T22:56:10.313894+0000","last_clean":"2024-09-14T22:56:10.313894+0000","last_became_active":"2024-09-14T22:55:51.143629+0000","last_became_peered":"2024-09-14T22:55:51.143629+0000","last_unstale":"2024-09-14T22:56:10.313894+0000","last_undegraded":"2024-09-14T22:56:10.313894+0000","last_fullsized":"2024-09-14T22:56:10.313894+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:51.928400+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.037412189999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":36,"num_read":3234,"num_read_kb":8551,"num_write":522,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"160'689","reported_seq":4144,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313011+0000","last_change":"2024-09-14T22:55:36.056675+0000","last_active":"2024-09-14T22:56:10.313011+0000","last_peered":"2024-09-14T22:56:10.313011+0000","last_clean":"2024-09-14T22:56:10.313011+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:56:10.313011+0000","last_undegraded":"2024-09-14T22:56:10.313011+0000","last_fullsized":"2024-09-14T22:56:10.313011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:48:33.776316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001452467,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5954,"num_read_kb":20565,"num_write":618,"num_write_kb":24613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"161'577","reported_seq":5751,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313749+0000","last_change":"2024-09-14T22:55:36.057396+0000","last_active":"2024-09-14T22:56:10.313749+0000","last_peered":"2024-09-14T22:56:10.313749+0000","last_clean":"2024-09-14T22:56:10.313749+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:56:10.313749+0000","last_undegraded":"2024-09-14T22:56:10.313749+0000","last_fullsized":"2024-09-14T22:56:10.313749+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:58.321946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000939155,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":6243,"num_read_kb":22048,"num_write":553,"num_write_kb":16971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"162'626","reported_seq":7317,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837797+0000","last_change":"2024-09-14T22:55:36.135696+0000","last_active":"2024-09-14T22:56:09.837797+0000","last_peered":"2024-09-14T22:56:09.837797+0000","last_clean":"2024-09-14T22:56:09.837797+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:56:09.837797+0000","last_undegraded":"2024-09-14T22:56:09.837797+0000","last_fullsized":"2024-09-14T22:56:09.837797+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:40:57.110923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014953080000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":8194,"num_read_kb":36336,"num_write":770,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"170'515","reported_seq":7151,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.838025+0000","last_change":"2024-09-14T22:55:51.374445+0000","last_active":"2024-09-14T22:56:09.838025+0000","last_peered":"2024-09-14T22:56:09.838025+0000","last_clean":"2024-09-14T22:56:09.838025+0000","last_became_active":"2024-09-14T22:55:51.141871+0000","last_became_peered":"2024-09-14T22:55:51.141871+0000","last_unstale":"2024-09-14T22:56:09.838025+0000","last_undegraded":"2024-09-14T22:56:09.838025+0000","last_fullsized":"2024-09-14T22:56:09.838025+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:57:25.784874+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.053472498,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":7727,"num_read_kb":40328,"num_write":448,"num_write_kb":21333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"170'348","reported_seq":6502,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837958+0000","last_change":"2024-09-14T22:55:51.268238+0000","last_active":"2024-09-14T22:56:09.837958+0000","last_peered":"2024-09-14T22:56:09.837958+0000","last_clean":"2024-09-14T22:56:09.837958+0000","last_became_active":"2024-09-14T22:55:51.141931+0000","last_became_peered":"2024-09-14T22:55:51.141931+0000","last_unstale":"2024-09-14T22:56:09.837958+0000","last_undegraded":"2024-09-14T22:56:09.837958+0000","last_fullsized":"2024-09-14T22:56:09.837958+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:22.125200+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.047107007999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5750,"num_read_kb":21575,"num_write":279,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'550","reported_seq":10161,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:10.313774+0000","last_change":"2024-09-14T22:55:36.062330+0000","last_active":"2024-09-14T22:56:10.313774+0000","last_peered":"2024-09-14T22:56:10.313774+0000","last_clean":"2024-09-14T22:56:10.313774+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:56:10.313774+0000","last_undegraded":"2024-09-14T22:56:10.313774+0000","last_fullsized":"2024-09-14T22:56:10.313774+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:09:02.042762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128364,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":11616,"num_read_kb":30610,"num_write":481,"num_write_kb":42134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"170'613","reported_seq":5711,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837789+0000","last_change":"2024-09-14T22:55:51.388404+0000","last_active":"2024-09-14T22:56:09.837789+0000","last_peered":"2024-09-14T22:56:09.837789+0000","last_clean":"2024-09-14T22:56:09.837789+0000","last_became_active":"2024-09-14T22:55:51.142779+0000","last_became_peered":"2024-09-14T22:55:51.142779+0000","last_unstale":"2024-09-14T22:56:09.837789+0000","last_undegraded":"2024-09-14T22:56:09.837789+0000","last_fullsized":"2024-09-14T22:56:09.837789+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:52:34.034047+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.041728379000000003,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":6204,"num_read_kb":34738,"num_write":712,"num_write_kb":24203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"161'410","reported_seq":1626,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.836322+0000","last_change":"2024-09-14T22:55:56.769064+0000","last_active":"2024-09-14T22:56:09.836322+0000","last_peered":"2024-09-14T22:56:09.836322+0000","last_clean":"2024-09-14T22:56:09.836322+0000","last_became_active":"2024-09-14T22:55:56.767773+0000","last_became_peered":"2024-09-14T22:55:56.767773+0000","last_unstale":"2024-09-14T22:56:09.836322+0000","last_undegraded":"2024-09-14T22:56:09.836322+0000","last_fullsized":"2024-09-14T22:56:09.836322+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T22:58:58.130702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013471519999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":1274,"num_read_kb":10604,"num_write":428,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"170'983","reported_seq":6790,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:09.837827+0000","last_change":"2024-09-14T22:55:51.397026+0000","last_active":"2024-09-14T22:56:09.837827+0000","last_peered":"2024-09-14T22:56:09.837827+0000","last_clean":"2024-09-14T22:56:09.837827+0000","last_became_active":"2024-09-14T22:55:51.143935+0000","last_became_peered":"2024-09-14T22:55:51.143935+0000","last_unstale":"2024-09-14T22:56:09.837827+0000","last_undegraded":"2024-09-14T22:56:09.837827+0000","last_fullsized":"2024-09-14T22:56:09.837827+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:53:05.734283+0000","last_clean_scrub_stamp":"2024-09-14T22:53:05.734283+0000","objects_scrubbed":0,"log_size":983,"log_dups_size":0,"ondisk_log_size":983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:40:42.066436+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.034758323000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":8863,"num_read_kb":12560,"num_write":1145,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":1377,"num_read":260757,"num_read_kb":1066701,"num_write":19012,"num_write_kb":702896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":141629825,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},"log_size":18820,"ondisk_log_size":18820,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56204,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315636,"statfs":{"total":96636764160,"available":96579211264,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29252,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342588,"statfs":{"total":96636764160,"available":96606810112,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51464,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320376,"statfs":{"total":96636764160,"available":96584065024,"internally_reserved":0,"allocated":2281472,"data_stored":1113541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":169,"seq":725849473031,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35788,"kb_used_data":2228,"kb_used_omap":47,"kb_used_meta":33488,"kb_avail":94336052,"statfs":{"total":96636764160,"available":96600117248,"internally_reserved":0,"allocated":2281472,"data_stored":1113541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49036,"internal_metadata":34291828},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41732,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330108,"statfs":{"total":96636764160,"available":96594030592,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542192,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29264,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342576,"statfs":{"total":96636764160,"available":96606797824,"internally_reserved":0,"allocated":2285568,"data_stored":1113560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56208,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315632,"statfs":{"total":96636764160,"available":96579207168,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:56:18.399 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.399 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.400 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.400 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.400 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.400 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.400 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.400 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.400 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.400 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T22:52:56.511969+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=56, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-14T22:52:56.511969+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=56, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T22:52:56.511969+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=56, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.401 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-14T22:52:56.511969+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=56, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T22:52:56.511969+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=56, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-14T22:52:56.511969+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=56, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-14T22:52:56.511969+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=56, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T22:52:56.511969+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=56, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.402 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-14T22:52:51.486958+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=52, tm_sec=51, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.403 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.404 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.404 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T22:53:05.734283+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=53, tm_sec=5, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=22, tm_min=56, tm_sec=13, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T22:56:18.404 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-14T22:56:38.405 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T22:56:38.695 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-14T22:56:38.696 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-14T22:56:38.708 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-14T22:56:36.824840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":1385,"num_object_clones":0,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":1377,"num_read":260873,"num_read_kb":1066801,"num_write":19173,"num_write_kb":705314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":141629863,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20325,"ondisk_log_size":20325,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":341592,"kb_used_data":18968,"kb_used_omap":58,"kb_used_meta":322437,"kb_avail":754633128,"statfs":{"total":773094113280,"available":772744323072,"internally_reserved":0,"allocated":19423232,"data_stored":10088988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":60192,"internal_metadata":330175712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002834"},"pg_stats":[{"pgid":"3.1f","version":"189'717","reported_seq":8475,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:15.127549+0000","last_change":"2024-09-14T22:56:15.127589+0000","last_active":"2024-09-14T22:56:15.127549+0000","last_peered":"2024-09-14T22:56:15.127549+0000","last_clean":"2024-09-14T22:56:15.127549+0000","last_became_active":"2024-09-14T22:53:06.757911+0000","last_became_peered":"2024-09-14T22:53:06.757911+0000","last_unstale":"2024-09-14T22:56:15.127549+0000","last_undegraded":"2024-09-14T22:56:15.127549+0000","last_fullsized":"2024-09-14T22:56:15.127549+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'717","last_scrub_stamp":"2024-09-14T22:56:15.127475+0000","last_deep_scrub":"189'717","last_deep_scrub_stamp":"2024-09-14T22:56:15.127475+0000","last_clean_scrub_stamp":"2024-09-14T22:56:15.127475+0000","objects_scrubbed":38,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:21:51.129920+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.001547748,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":9382,"num_read_kb":31310,"num_write":714,"num_write_kb":7692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"189'568","reported_seq":8668,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:20.098674+0000","last_change":"2024-09-14T22:56:20.098674+0000","last_active":"2024-09-14T22:56:20.098674+0000","last_peered":"2024-09-14T22:56:20.098674+0000","last_clean":"2024-09-14T22:56:20.098674+0000","last_became_active":"2024-09-14T22:54:28.172320+0000","last_became_peered":"2024-09-14T22:54:28.172320+0000","last_unstale":"2024-09-14T22:56:20.098674+0000","last_undegraded":"2024-09-14T22:56:20.098674+0000","last_fullsized":"2024-09-14T22:56:20.098674+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'568","last_scrub_stamp":"2024-09-14T22:56:20.098559+0000","last_deep_scrub":"189'568","last_deep_scrub_stamp":"2024-09-14T22:56:20.098559+0000","last_clean_scrub_stamp":"2024-09-14T22:56:20.098559+0000","objects_scrubbed":39,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:29:23.293702+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0012941300000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":8409,"num_read_kb":20232,"num_write":463,"num_write_kb":12541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":6099649,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"189'566","reported_seq":9341,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:23.763024+0000","last_change":"2024-09-14T22:56:23.763076+0000","last_active":"2024-09-14T22:56:23.763024+0000","last_peered":"2024-09-14T22:56:23.763024+0000","last_clean":"2024-09-14T22:56:23.763024+0000","last_became_active":"2024-09-14T22:55:51.142689+0000","last_became_peered":"2024-09-14T22:55:51.142689+0000","last_unstale":"2024-09-14T22:56:23.763024+0000","last_undegraded":"2024-09-14T22:56:23.763024+0000","last_fullsized":"2024-09-14T22:56:23.763024+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'566","last_scrub_stamp":"2024-09-14T22:56:23.762928+0000","last_deep_scrub":"189'566","last_deep_scrub_stamp":"2024-09-14T22:56:23.762928+0000","last_clean_scrub_stamp":"2024-09-14T22:56:23.762928+0000","objects_scrubbed":45,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:12:17.770429+0000","scrub_duration":33,"objects_trimmed":28,"snaptrim_duration":0.057378967000000003,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":9221,"num_read_kb":38644,"num_write":441,"num_write_kb":52016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"189'684","reported_seq":3663,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:24.734684+0000","last_change":"2024-09-14T22:56:24.734726+0000","last_active":"2024-09-14T22:56:24.734684+0000","last_peered":"2024-09-14T22:56:24.734684+0000","last_clean":"2024-09-14T22:56:24.734684+0000","last_became_active":"2024-09-14T22:55:51.144955+0000","last_became_peered":"2024-09-14T22:55:51.144955+0000","last_unstale":"2024-09-14T22:56:24.734684+0000","last_undegraded":"2024-09-14T22:56:24.734684+0000","last_fullsized":"2024-09-14T22:56:24.734684+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'684","last_scrub_stamp":"2024-09-14T22:56:24.734598+0000","last_deep_scrub":"189'684","last_deep_scrub_stamp":"2024-09-14T22:56:24.734598+0000","last_clean_scrub_stamp":"2024-09-14T22:56:24.734598+0000","objects_scrubbed":35,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:58:32.882420+0000","scrub_duration":27,"objects_trimmed":14,"snaptrim_duration":0.037609783000000001,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":34,"num_read":4493,"num_read_kb":12183,"num_write":718,"num_write_kb":8978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"189'381","reported_seq":5426,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:17.082845+0000","last_change":"2024-09-14T22:56:17.082905+0000","last_active":"2024-09-14T22:56:17.082845+0000","last_peered":"2024-09-14T22:56:17.082845+0000","last_clean":"2024-09-14T22:56:17.082845+0000","last_became_active":"2024-09-14T22:55:51.137980+0000","last_became_peered":"2024-09-14T22:55:51.137980+0000","last_unstale":"2024-09-14T22:56:17.082845+0000","last_undegraded":"2024-09-14T22:56:17.082845+0000","last_fullsized":"2024-09-14T22:56:17.082845+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'381","last_scrub_stamp":"2024-09-14T22:56:17.082764+0000","last_deep_scrub":"189'381","last_deep_scrub_stamp":"2024-09-14T22:56:17.082764+0000","last_clean_scrub_stamp":"2024-09-14T22:56:17.082764+0000","objects_scrubbed":41,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:47:04.368669+0000","scrub_duration":25,"objects_trimmed":16,"snaptrim_duration":0.034914117000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":4616,"num_read_kb":54005,"num_write":312,"num_write_kb":18747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"189'737","reported_seq":14610,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:20.041348+0000","last_change":"2024-09-14T22:56:20.041390+0000","last_active":"2024-09-14T22:56:20.041348+0000","last_peered":"2024-09-14T22:56:20.041348+0000","last_clean":"2024-09-14T22:56:20.041348+0000","last_became_active":"2024-09-14T22:55:53.765594+0000","last_became_peered":"2024-09-14T22:55:53.765594+0000","last_unstale":"2024-09-14T22:56:20.041348+0000","last_undegraded":"2024-09-14T22:56:20.041348+0000","last_fullsized":"2024-09-14T22:56:20.041348+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'737","last_scrub_stamp":"2024-09-14T22:56:20.041266+0000","last_deep_scrub":"189'737","last_deep_scrub_stamp":"2024-09-14T22:56:20.041266+0000","last_clean_scrub_stamp":"2024-09-14T22:56:20.041266+0000","objects_scrubbed":46,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:19:02.031620+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0.00071064000000000001,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":16209,"num_read_kb":53499,"num_write":537,"num_write_kb":29099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":18306298,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"189'538","reported_seq":16239,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:18.020390+0000","last_change":"2024-09-14T22:56:18.020439+0000","last_active":"2024-09-14T22:56:18.020390+0000","last_peered":"2024-09-14T22:56:18.020390+0000","last_clean":"2024-09-14T22:56:18.020390+0000","last_became_active":"2024-09-14T22:55:58.778624+0000","last_became_peered":"2024-09-14T22:55:58.778624+0000","last_unstale":"2024-09-14T22:56:18.020390+0000","last_undegraded":"2024-09-14T22:56:18.020390+0000","last_fullsized":"2024-09-14T22:56:18.020390+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'538","last_scrub_stamp":"2024-09-14T22:56:18.020292+0000","last_deep_scrub":"189'538","last_deep_scrub_stamp":"2024-09-14T22:56:18.020292+0000","last_clean_scrub_stamp":"2024-09-14T22:56:18.020292+0000","objects_scrubbed":53,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:00:09.592573+0000","scrub_duration":39,"objects_trimmed":2,"snaptrim_duration":0.0059748170000000003,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":15953,"num_read_kb":92561,"num_write":414,"num_write_kb":38470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"189'726","reported_seq":2511,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:19.072503+0000","last_change":"2024-09-14T22:56:19.072548+0000","last_active":"2024-09-14T22:56:19.072503+0000","last_peered":"2024-09-14T22:56:19.072503+0000","last_clean":"2024-09-14T22:56:19.072503+0000","last_became_active":"2024-09-14T22:54:28.170911+0000","last_became_peered":"2024-09-14T22:54:28.170911+0000","last_unstale":"2024-09-14T22:56:19.072503+0000","last_undegraded":"2024-09-14T22:56:19.072503+0000","last_fullsized":"2024-09-14T22:56:19.072503+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'726","last_scrub_stamp":"2024-09-14T22:56:19.072404+0000","last_deep_scrub":"189'726","last_deep_scrub_stamp":"2024-09-14T22:56:19.072404+0000","last_clean_scrub_stamp":"2024-09-14T22:56:19.072404+0000","objects_scrubbed":51,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:41:01.271061+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0.001629828,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":2959,"num_read_kb":22740,"num_write":861,"num_write_kb":10481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":8586311,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"189'689","reported_seq":6971,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:20.074585+0000","last_change":"2024-09-14T22:56:20.074657+0000","last_active":"2024-09-14T22:56:20.074585+0000","last_peered":"2024-09-14T22:56:20.074585+0000","last_clean":"2024-09-14T22:56:20.074585+0000","last_became_active":"2024-09-14T22:55:56.768221+0000","last_became_peered":"2024-09-14T22:55:56.768221+0000","last_unstale":"2024-09-14T22:56:20.074585+0000","last_undegraded":"2024-09-14T22:56:20.074585+0000","last_fullsized":"2024-09-14T22:56:20.074585+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'689","last_scrub_stamp":"2024-09-14T22:56:20.074471+0000","last_deep_scrub":"189'689","last_deep_scrub_stamp":"2024-09-14T22:56:20.074471+0000","last_clean_scrub_stamp":"2024-09-14T22:56:20.074471+0000","objects_scrubbed":37,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:34:14.558120+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0.001569408,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":7874,"num_read_kb":34799,"num_write":817,"num_write_kb":27727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"189'472","reported_seq":9827,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:18.742727+0000","last_change":"2024-09-14T22:56:18.742775+0000","last_active":"2024-09-14T22:56:18.742727+0000","last_peered":"2024-09-14T22:56:18.742727+0000","last_clean":"2024-09-14T22:56:18.742727+0000","last_became_active":"2024-09-14T22:53:06.754626+0000","last_became_peered":"2024-09-14T22:53:06.754626+0000","last_unstale":"2024-09-14T22:56:18.742727+0000","last_undegraded":"2024-09-14T22:56:18.742727+0000","last_fullsized":"2024-09-14T22:56:18.742727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'472","last_scrub_stamp":"2024-09-14T22:56:18.742634+0000","last_deep_scrub":"189'472","last_deep_scrub_stamp":"2024-09-14T22:56:18.742634+0000","last_clean_scrub_stamp":"2024-09-14T22:56:18.742634+0000","objects_scrubbed":39,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:12:58.642386+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0011312640000000001,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10352,"num_read_kb":46008,"num_write":347,"num_write_kb":32677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"189'460","reported_seq":9471,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:20.927504+0000","last_change":"2024-09-14T22:56:20.927553+0000","last_active":"2024-09-14T22:56:20.927504+0000","last_peered":"2024-09-14T22:56:20.927504+0000","last_clean":"2024-09-14T22:56:20.927504+0000","last_became_active":"2024-09-14T22:53:06.761695+0000","last_became_peered":"2024-09-14T22:53:06.761695+0000","last_unstale":"2024-09-14T22:56:20.927504+0000","last_undegraded":"2024-09-14T22:56:20.927504+0000","last_fullsized":"2024-09-14T22:56:20.927504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'460","last_scrub_stamp":"2024-09-14T22:56:20.927422+0000","last_deep_scrub":"189'460","last_deep_scrub_stamp":"2024-09-14T22:56:20.927422+0000","last_clean_scrub_stamp":"2024-09-14T22:56:20.927422+0000","objects_scrubbed":39,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:27:32.057306+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.00086910600000000002,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":10004,"num_read_kb":43389,"num_write":402,"num_write_kb":13562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"189'644","reported_seq":10968,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:18.115210+0000","last_change":"2024-09-14T22:56:18.115301+0000","last_active":"2024-09-14T22:56:18.115210+0000","last_peered":"2024-09-14T22:56:18.115210+0000","last_clean":"2024-09-14T22:56:18.115210+0000","last_became_active":"2024-09-14T22:55:51.138619+0000","last_became_peered":"2024-09-14T22:55:51.138619+0000","last_unstale":"2024-09-14T22:56:18.115210+0000","last_undegraded":"2024-09-14T22:56:18.115210+0000","last_fullsized":"2024-09-14T22:56:18.115210+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'644","last_scrub_stamp":"2024-09-14T22:56:18.115122+0000","last_deep_scrub":"189'644","last_deep_scrub_stamp":"2024-09-14T22:56:18.115122+0000","last_clean_scrub_stamp":"2024-09-14T22:56:18.115122+0000","objects_scrubbed":43,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T04:23:07.472076+0000","scrub_duration":33,"objects_trimmed":32,"snaptrim_duration":0.074291557999999994,"stat_sum":{"num_bytes":0,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":43,"num_read":11380,"num_read_kb":66402,"num_write":623,"num_write_kb":37521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":361,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:18.536853+0000","last_change":"2024-09-14T22:56:18.536887+0000","last_active":"2024-09-14T22:56:18.536853+0000","last_peered":"2024-09-14T22:56:18.536853+0000","last_clean":"2024-09-14T22:56:18.536853+0000","last_became_active":"2024-09-14T22:52:57.533281+0000","last_became_peered":"2024-09-14T22:52:57.533281+0000","last_unstale":"2024-09-14T22:56:18.536853+0000","last_undegraded":"2024-09-14T22:56:18.536853+0000","last_fullsized":"2024-09-14T22:56:18.536853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:56:18.536759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:56:18.536759+0000","last_clean_scrub_stamp":"2024-09-14T22:56:18.536759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:28:27.148438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045116600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"189'764","reported_seq":8112,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:16.109677+0000","last_change":"2024-09-14T22:56:16.109744+0000","last_active":"2024-09-14T22:56:16.109677+0000","last_peered":"2024-09-14T22:56:16.109677+0000","last_clean":"2024-09-14T22:56:16.109677+0000","last_became_active":"2024-09-14T22:55:32.040243+0000","last_became_peered":"2024-09-14T22:55:32.040243+0000","last_unstale":"2024-09-14T22:56:16.109677+0000","last_undegraded":"2024-09-14T22:56:16.109677+0000","last_fullsized":"2024-09-14T22:56:16.109677+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'764","last_scrub_stamp":"2024-09-14T22:56:16.109598+0000","last_deep_scrub":"189'764","last_deep_scrub_stamp":"2024-09-14T22:56:16.109598+0000","last_clean_scrub_stamp":"2024-09-14T22:56:16.109598+0000","objects_scrubbed":36,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:58:52.357612+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0.0012928849999999999,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":9421,"num_read_kb":42199,"num_write":751,"num_write_kb":24667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":319,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:19.708561+0000","last_change":"2024-09-14T22:56:19.708614+0000","last_active":"2024-09-14T22:56:19.708561+0000","last_peered":"2024-09-14T22:56:19.708561+0000","last_clean":"2024-09-14T22:56:19.708561+0000","last_became_active":"2024-09-14T22:53:21.943936+0000","last_became_peered":"2024-09-14T22:53:21.943936+0000","last_unstale":"2024-09-14T22:56:19.708561+0000","last_undegraded":"2024-09-14T22:56:19.708561+0000","last_fullsized":"2024-09-14T22:56:19.708561+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:56:19.708455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:56:19.708455+0000","last_clean_scrub_stamp":"2024-09-14T22:56:19.708455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T06:46:04.813688+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00044360599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"189'372","reported_seq":2173,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:21.966447+0000","last_change":"2024-09-14T22:56:21.966495+0000","last_active":"2024-09-14T22:56:21.966447+0000","last_peered":"2024-09-14T22:56:21.966447+0000","last_clean":"2024-09-14T22:56:21.966447+0000","last_became_active":"2024-09-14T22:54:07.043262+0000","last_became_peered":"2024-09-14T22:54:07.043262+0000","last_unstale":"2024-09-14T22:56:21.966447+0000","last_undegraded":"2024-09-14T22:56:21.966447+0000","last_fullsized":"2024-09-14T22:56:21.966447+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'372","last_scrub_stamp":"2024-09-14T22:56:21.966331+0000","last_deep_scrub":"189'372","last_deep_scrub_stamp":"2024-09-14T22:56:21.966331+0000","last_clean_scrub_stamp":"2024-09-14T22:56:21.966331+0000","objects_scrubbed":37,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:13.938140+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.00079363000000000003,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":37,"num_read":1234,"num_read_kb":15244,"num_write":279,"num_write_kb":17581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6267304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":366,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:24.925953+0000","last_change":"2024-09-14T22:56:24.925977+0000","last_active":"2024-09-14T22:56:24.925953+0000","last_peered":"2024-09-14T22:56:24.925953+0000","last_clean":"2024-09-14T22:56:24.925953+0000","last_became_active":"2024-09-14T22:55:39.451421+0000","last_became_peered":"2024-09-14T22:55:39.451421+0000","last_unstale":"2024-09-14T22:56:24.925953+0000","last_undegraded":"2024-09-14T22:56:24.925953+0000","last_fullsized":"2024-09-14T22:56:24.925953+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:56:24.925846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:56:24.925846+0000","last_clean_scrub_stamp":"2024-09-14T22:56:24.925846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:28:47.957107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000428486,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"189'889","reported_seq":14352,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:16.977132+0000","last_change":"2024-09-14T22:56:16.977189+0000","last_active":"2024-09-14T22:56:16.977132+0000","last_peered":"2024-09-14T22:56:16.977132+0000","last_clean":"2024-09-14T22:56:16.977132+0000","last_became_active":"2024-09-14T22:55:58.776278+0000","last_became_peered":"2024-09-14T22:55:58.776278+0000","last_unstale":"2024-09-14T22:56:16.977132+0000","last_undegraded":"2024-09-14T22:56:16.977132+0000","last_fullsized":"2024-09-14T22:56:16.977132+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'889","last_scrub_stamp":"2024-09-14T22:56:16.977044+0000","last_deep_scrub":"189'889","last_deep_scrub_stamp":"2024-09-14T22:56:16.977044+0000","last_clean_scrub_stamp":"2024-09-14T22:56:16.977044+0000","objects_scrubbed":45,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:38:24.876113+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.0019992360000000002,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":15582,"num_read_kb":40098,"num_write":1081,"num_write_kb":19732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":15786495,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":80,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:15.935617+0000","last_change":"2024-09-14T22:56:15.935651+0000","last_active":"2024-09-14T22:56:15.935617+0000","last_peered":"2024-09-14T22:56:15.935617+0000","last_clean":"2024-09-14T22:56:15.935617+0000","last_became_active":"2024-09-14T22:55:58.776284+0000","last_became_peered":"2024-09-14T22:55:58.776284+0000","last_unstale":"2024-09-14T22:56:15.935617+0000","last_undegraded":"2024-09-14T22:56:15.935617+0000","last_fullsized":"2024-09-14T22:56:15.935617+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:56:15.935500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:56:15.935500+0000","last_clean_scrub_stamp":"2024-09-14T22:56:15.935500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:55:49.186455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"189'475","reported_seq":15841,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:21.698142+0000","last_change":"2024-09-14T22:56:21.698184+0000","last_active":"2024-09-14T22:56:21.698142+0000","last_peered":"2024-09-14T22:56:21.698142+0000","last_clean":"2024-09-14T22:56:21.698142+0000","last_became_active":"2024-09-14T22:54:28.176866+0000","last_became_peered":"2024-09-14T22:54:28.176866+0000","last_unstale":"2024-09-14T22:56:21.698142+0000","last_undegraded":"2024-09-14T22:56:21.698142+0000","last_fullsized":"2024-09-14T22:56:21.698142+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'475","last_scrub_stamp":"2024-09-14T22:56:21.698045+0000","last_deep_scrub":"189'475","last_deep_scrub_stamp":"2024-09-14T22:56:21.698045+0000","last_clean_scrub_stamp":"2024-09-14T22:56:21.698045+0000","objects_scrubbed":41,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:52:20.680224+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0.00055810799999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14507,"num_read_kb":48865,"num_write":357,"num_write_kb":19125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":21851623,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"189'1052","reported_seq":6904,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:17.699091+0000","last_change":"2024-09-14T22:56:17.699117+0000","last_active":"2024-09-14T22:56:17.699091+0000","last_peered":"2024-09-14T22:56:17.699091+0000","last_clean":"2024-09-14T22:56:17.699091+0000","last_became_active":"2024-09-14T22:54:47.747387+0000","last_became_peered":"2024-09-14T22:54:47.747387+0000","last_unstale":"2024-09-14T22:56:17.699091+0000","last_undegraded":"2024-09-14T22:56:17.699091+0000","last_fullsized":"2024-09-14T22:56:17.699091+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'1052","last_scrub_stamp":"2024-09-14T22:56:17.699018+0000","last_deep_scrub":"189'1052","last_deep_scrub_stamp":"2024-09-14T22:56:17.699018+0000","last_clean_scrub_stamp":"2024-09-14T22:56:17.699018+0000","objects_scrubbed":51,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:21:29.856087+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0.0016356700000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":50,"num_read":9071,"num_read_kb":57942,"num_write":1041,"num_write_kb":41152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":29933802,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"189'3","reported_seq":381,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:22.690524+0000","last_change":"2024-09-14T22:56:22.690555+0000","last_active":"2024-09-14T22:56:22.690524+0000","last_peered":"2024-09-14T22:56:22.690524+0000","last_clean":"2024-09-14T22:56:22.690524+0000","last_became_active":"2024-09-14T22:55:32.040648+0000","last_became_peered":"2024-09-14T22:55:32.040648+0000","last_unstale":"2024-09-14T22:56:22.690524+0000","last_undegraded":"2024-09-14T22:56:22.690524+0000","last_fullsized":"2024-09-14T22:56:22.690524+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'3","last_scrub_stamp":"2024-09-14T22:56:22.690432+0000","last_deep_scrub":"189'3","last_deep_scrub_stamp":"2024-09-14T22:56:22.690432+0000","last_clean_scrub_stamp":"2024-09-14T22:56:22.690432+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:28:28.198367+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010351729999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"189'403","reported_seq":1951,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:20.586331+0000","last_change":"2024-09-14T22:56:20.586365+0000","last_active":"2024-09-14T22:56:20.586331+0000","last_peered":"2024-09-14T22:56:20.586331+0000","last_clean":"2024-09-14T22:56:20.586331+0000","last_became_active":"2024-09-14T22:53:08.138972+0000","last_became_peered":"2024-09-14T22:53:08.138972+0000","last_unstale":"2024-09-14T22:56:20.586331+0000","last_undegraded":"2024-09-14T22:56:20.586331+0000","last_fullsized":"2024-09-14T22:56:20.586331+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'403","last_scrub_stamp":"2024-09-14T22:56:20.586227+0000","last_deep_scrub":"189'403","last_deep_scrub_stamp":"2024-09-14T22:56:20.586227+0000","last_clean_scrub_stamp":"2024-09-14T22:56:20.586227+0000","objects_scrubbed":33,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T05:20:07.847134+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0.001140828,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":1390,"num_read_kb":9489,"num_write":336,"num_write_kb":12050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":372,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:16.618818+0000","last_change":"2024-09-14T22:56:16.618837+0000","last_active":"2024-09-14T22:56:16.618818+0000","last_peered":"2024-09-14T22:56:16.618818+0000","last_clean":"2024-09-14T22:56:16.618818+0000","last_became_active":"2024-09-14T22:54:25.160431+0000","last_became_peered":"2024-09-14T22:54:25.160431+0000","last_unstale":"2024-09-14T22:56:16.618818+0000","last_undegraded":"2024-09-14T22:56:16.618818+0000","last_fullsized":"2024-09-14T22:56:16.618818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:56:16.618726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:56:16.618726+0000","last_clean_scrub_stamp":"2024-09-14T22:56:16.618726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:14:12.696262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033082800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"189'673","reported_seq":9443,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:18.066865+0000","last_change":"2024-09-14T22:56:18.066897+0000","last_active":"2024-09-14T22:56:18.066865+0000","last_peered":"2024-09-14T22:56:18.066865+0000","last_clean":"2024-09-14T22:56:18.066865+0000","last_became_active":"2024-09-14T22:55:51.139441+0000","last_became_peered":"2024-09-14T22:55:51.139441+0000","last_unstale":"2024-09-14T22:56:18.066865+0000","last_undegraded":"2024-09-14T22:56:18.066865+0000","last_fullsized":"2024-09-14T22:56:18.066865+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'673","last_scrub_stamp":"2024-09-14T22:56:18.066797+0000","last_deep_scrub":"189'673","last_deep_scrub_stamp":"2024-09-14T22:56:18.066797+0000","last_clean_scrub_stamp":"2024-09-14T22:56:18.066797+0000","objects_scrubbed":41,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T10:04:57.332973+0000","scrub_duration":25,"objects_trimmed":16,"snaptrim_duration":0.026769095999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":10199,"num_read_kb":21422,"num_write":735,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:23.893968+0000","last_change":"2024-09-14T22:56:23.893994+0000","last_active":"2024-09-14T22:56:23.893968+0000","last_peered":"2024-09-14T22:56:23.893968+0000","last_clean":"2024-09-14T22:56:23.893968+0000","last_became_active":"2024-09-14T22:55:32.039728+0000","last_became_peered":"2024-09-14T22:55:32.039728+0000","last_unstale":"2024-09-14T22:56:23.893968+0000","last_undegraded":"2024-09-14T22:56:23.893968+0000","last_fullsized":"2024-09-14T22:56:23.893968+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T22:56:23.893859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T22:56:23.893859+0000","last_clean_scrub_stamp":"2024-09-14T22:56:23.893859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:58:17.842763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045427400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"189'544","reported_seq":7010,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:19.088751+0000","last_change":"2024-09-14T22:56:19.088788+0000","last_active":"2024-09-14T22:56:19.088751+0000","last_peered":"2024-09-14T22:56:19.088751+0000","last_clean":"2024-09-14T22:56:19.088751+0000","last_became_active":"2024-09-14T22:55:51.141016+0000","last_became_peered":"2024-09-14T22:55:51.141016+0000","last_unstale":"2024-09-14T22:56:19.088751+0000","last_undegraded":"2024-09-14T22:56:19.088751+0000","last_fullsized":"2024-09-14T22:56:19.088751+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'544","last_scrub_stamp":"2024-09-14T22:56:19.088670+0000","last_deep_scrub":"189'544","last_deep_scrub_stamp":"2024-09-14T22:56:19.088670+0000","last_clean_scrub_stamp":"2024-09-14T22:56:19.088670+0000","objects_scrubbed":57,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:39:58.160565+0000","scrub_duration":42,"objects_trimmed":22,"snaptrim_duration":0.063591116000000003,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":57,"num_read":6223,"num_read_kb":32196,"num_write":426,"num_write_kb":41584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":11547701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"189'2","reported_seq":362,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:17.755378+0000","last_change":"2024-09-14T22:56:17.755409+0000","last_active":"2024-09-14T22:56:17.755378+0000","last_peered":"2024-09-14T22:56:17.755378+0000","last_clean":"2024-09-14T22:56:17.755378+0000","last_became_active":"2024-09-14T22:52:57.530139+0000","last_became_peered":"2024-09-14T22:52:57.530139+0000","last_unstale":"2024-09-14T22:56:17.755378+0000","last_undegraded":"2024-09-14T22:56:17.755378+0000","last_fullsized":"2024-09-14T22:56:17.755378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'2","last_scrub_stamp":"2024-09-14T22:56:17.755283+0000","last_deep_scrub":"189'2","last_deep_scrub_stamp":"2024-09-14T22:56:17.755283+0000","last_clean_scrub_stamp":"2024-09-14T22:56:17.755283+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:16:45.038885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00045670599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"189'119","reported_seq":514,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:18.920280+0000","last_change":"2024-09-14T22:56:18.920318+0000","last_active":"2024-09-14T22:56:18.920280+0000","last_peered":"2024-09-14T22:56:18.920280+0000","last_clean":"2024-09-14T22:56:18.920280+0000","last_became_active":"2024-09-14T22:52:52.653685+0000","last_became_peered":"2024-09-14T22:52:52.653685+0000","last_unstale":"2024-09-14T22:56:18.920280+0000","last_undegraded":"2024-09-14T22:56:18.920280+0000","last_fullsized":"2024-09-14T22:56:18.920280+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'119","last_scrub_stamp":"2024-09-14T22:56:18.920146+0000","last_deep_scrub":"189'119","last_deep_scrub_stamp":"2024-09-14T22:56:18.920146+0000","last_clean_scrub_stamp":"2024-09-14T22:56:18.920146+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:38:41.374914+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"189'987","reported_seq":10336,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:18.110978+0000","last_change":"2024-09-14T22:56:18.111059+0000","last_active":"2024-09-14T22:56:18.110978+0000","last_peered":"2024-09-14T22:56:18.110978+0000","last_clean":"2024-09-14T22:56:18.110978+0000","last_became_active":"2024-09-14T22:53:48.924399+0000","last_became_peered":"2024-09-14T22:53:48.924399+0000","last_unstale":"2024-09-14T22:56:18.110978+0000","last_undegraded":"2024-09-14T22:56:18.110978+0000","last_fullsized":"2024-09-14T22:56:18.110978+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'987","last_scrub_stamp":"2024-09-14T22:56:18.110896+0000","last_deep_scrub":"189'987","last_deep_scrub_stamp":"2024-09-14T22:56:18.110896+0000","last_clean_scrub_stamp":"2024-09-14T22:56:18.110896+0000","objects_scrubbed":47,"log_size":987,"log_dups_size":0,"ondisk_log_size":987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:21:40.646054+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0019587290000000002,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":14513,"num_read_kb":36136,"num_write":948,"num_write_kb":13116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":10974017,"num_keys_recovered":18,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"189'584","reported_seq":2341,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:17.098612+0000","last_change":"2024-09-14T22:56:17.098639+0000","last_active":"2024-09-14T22:56:17.098612+0000","last_peered":"2024-09-14T22:56:17.098612+0000","last_clean":"2024-09-14T22:56:17.098612+0000","last_became_active":"2024-09-14T22:55:51.135840+0000","last_became_peered":"2024-09-14T22:55:51.135840+0000","last_unstale":"2024-09-14T22:56:17.098612+0000","last_undegraded":"2024-09-14T22:56:17.098612+0000","last_fullsized":"2024-09-14T22:56:17.098612+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'584","last_scrub_stamp":"2024-09-14T22:56:17.098533+0000","last_deep_scrub":"189'584","last_deep_scrub_stamp":"2024-09-14T22:56:17.098533+0000","last_clean_scrub_stamp":"2024-09-14T22:56:17.098533+0000","objects_scrubbed":53,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:46:38.517053+0000","scrub_duration":36,"objects_trimmed":26,"snaptrim_duration":0.046042974,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":2706,"num_read_kb":9423,"num_write":453,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"189'568","reported_seq":2553,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:21.559430+0000","last_change":"2024-09-14T22:56:21.559477+0000","last_active":"2024-09-14T22:56:21.559430+0000","last_peered":"2024-09-14T22:56:21.559430+0000","last_clean":"2024-09-14T22:56:21.559430+0000","last_became_active":"2024-09-14T22:55:51.143629+0000","last_became_peered":"2024-09-14T22:55:51.143629+0000","last_unstale":"2024-09-14T22:56:21.559430+0000","last_undegraded":"2024-09-14T22:56:21.559430+0000","last_fullsized":"2024-09-14T22:56:21.559430+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'568","last_scrub_stamp":"2024-09-14T22:56:21.559342+0000","last_deep_scrub":"189'568","last_deep_scrub_stamp":"2024-09-14T22:56:21.559342+0000","last_clean_scrub_stamp":"2024-09-14T22:56:21.559342+0000","objects_scrubbed":37,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T00:02:45.476017+0000","scrub_duration":27,"objects_trimmed":16,"snaptrim_duration":0.037412189999999998,"stat_sum":{"num_bytes":0,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":36,"num_read":3234,"num_read_kb":8551,"num_write":522,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"189'738","reported_seq":4203,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:17.142959+0000","last_change":"2024-09-14T22:56:17.143020+0000","last_active":"2024-09-14T22:56:17.142959+0000","last_peered":"2024-09-14T22:56:17.142959+0000","last_clean":"2024-09-14T22:56:17.142959+0000","last_became_active":"2024-09-14T22:53:06.756632+0000","last_became_peered":"2024-09-14T22:53:06.756632+0000","last_unstale":"2024-09-14T22:56:17.142959+0000","last_undegraded":"2024-09-14T22:56:17.142959+0000","last_fullsized":"2024-09-14T22:56:17.142959+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'738","last_scrub_stamp":"2024-09-14T22:56:17.142887+0000","last_deep_scrub":"189'738","last_deep_scrub_stamp":"2024-09-14T22:56:17.142887+0000","last_clean_scrub_stamp":"2024-09-14T22:56:17.142887+0000","objects_scrubbed":49,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:23:20.898360+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0.001452467,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5954,"num_read_kb":20565,"num_write":618,"num_write_kb":24613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"189'625","reported_seq":5809,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:20.741506+0000","last_change":"2024-09-14T22:56:20.741549+0000","last_active":"2024-09-14T22:56:20.741506+0000","last_peered":"2024-09-14T22:56:20.741506+0000","last_clean":"2024-09-14T22:56:20.741506+0000","last_became_active":"2024-09-14T22:54:28.176695+0000","last_became_peered":"2024-09-14T22:54:28.176695+0000","last_unstale":"2024-09-14T22:56:20.741506+0000","last_undegraded":"2024-09-14T22:56:20.741506+0000","last_fullsized":"2024-09-14T22:56:20.741506+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'625","last_scrub_stamp":"2024-09-14T22:56:20.741411+0000","last_deep_scrub":"189'625","last_deep_scrub_stamp":"2024-09-14T22:56:20.741411+0000","last_clean_scrub_stamp":"2024-09-14T22:56:20.741411+0000","objects_scrubbed":48,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T03:29:52.249907+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0.000939155,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":6243,"num_read_kb":22048,"num_write":553,"num_write_kb":16971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":12276625,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"189'670","reported_seq":7371,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:19.963760+0000","last_change":"2024-09-14T22:56:19.963825+0000","last_active":"2024-09-14T22:56:19.963760+0000","last_peered":"2024-09-14T22:56:19.963760+0000","last_clean":"2024-09-14T22:56:19.963760+0000","last_became_active":"2024-09-14T22:53:06.758959+0000","last_became_peered":"2024-09-14T22:53:06.758959+0000","last_unstale":"2024-09-14T22:56:19.963760+0000","last_undegraded":"2024-09-14T22:56:19.963760+0000","last_fullsized":"2024-09-14T22:56:19.963760+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'670","last_scrub_stamp":"2024-09-14T22:56:19.963669+0000","last_deep_scrub":"189'670","last_deep_scrub_stamp":"2024-09-14T22:56:19.963669+0000","last_clean_scrub_stamp":"2024-09-14T22:56:19.963669+0000","objects_scrubbed":44,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:35:47.077868+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.0014953080000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":8194,"num_read_kb":36336,"num_write":770,"num_write_kb":6580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"189'554","reported_seq":7200,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:25.979879+0000","last_change":"2024-09-14T22:56:25.979931+0000","last_active":"2024-09-14T22:56:25.979879+0000","last_peered":"2024-09-14T22:56:25.979879+0000","last_clean":"2024-09-14T22:56:25.979879+0000","last_became_active":"2024-09-14T22:55:51.141871+0000","last_became_peered":"2024-09-14T22:55:51.141871+0000","last_unstale":"2024-09-14T22:56:25.979879+0000","last_undegraded":"2024-09-14T22:56:25.979879+0000","last_fullsized":"2024-09-14T22:56:25.979879+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'554","last_scrub_stamp":"2024-09-14T22:56:25.979789+0000","last_deep_scrub":"189'554","last_deep_scrub_stamp":"2024-09-14T22:56:25.979789+0000","last_clean_scrub_stamp":"2024-09-14T22:56:25.979789+0000","objects_scrubbed":39,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:43:07.052462+0000","scrub_duration":29,"objects_trimmed":24,"snaptrim_duration":0.053472498,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":7727,"num_read_kb":40328,"num_write":448,"num_write_kb":21333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"189'389","reported_seq":6553,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:22.944176+0000","last_change":"2024-09-14T22:56:22.944256+0000","last_active":"2024-09-14T22:56:22.944176+0000","last_peered":"2024-09-14T22:56:22.944176+0000","last_clean":"2024-09-14T22:56:22.944176+0000","last_became_active":"2024-09-14T22:55:51.141931+0000","last_became_peered":"2024-09-14T22:55:51.141931+0000","last_unstale":"2024-09-14T22:56:22.944176+0000","last_undegraded":"2024-09-14T22:56:22.944176+0000","last_fullsized":"2024-09-14T22:56:22.944176+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'389","last_scrub_stamp":"2024-09-14T22:56:22.944079+0000","last_deep_scrub":"189'389","last_deep_scrub_stamp":"2024-09-14T22:56:22.944079+0000","last_clean_scrub_stamp":"2024-09-14T22:56:22.944079+0000","objects_scrubbed":41,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T07:51:20.953819+0000","scrub_duration":30,"objects_trimmed":24,"snaptrim_duration":0.047107007999999999,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":5750,"num_read_kb":21575,"num_write":279,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"189'592","reported_seq":10213,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:19.562368+0000","last_change":"2024-09-14T22:56:19.562406+0000","last_active":"2024-09-14T22:56:19.562368+0000","last_peered":"2024-09-14T22:56:19.562368+0000","last_clean":"2024-09-14T22:56:19.562368+0000","last_became_active":"2024-09-14T22:53:06.756141+0000","last_became_peered":"2024-09-14T22:53:06.756141+0000","last_unstale":"2024-09-14T22:56:19.562368+0000","last_undegraded":"2024-09-14T22:56:19.562368+0000","last_fullsized":"2024-09-14T22:56:19.562368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'592","last_scrub_stamp":"2024-09-14T22:56:19.562288+0000","last_deep_scrub":"189'592","last_deep_scrub_stamp":"2024-09-14T22:56:19.562288+0000","last_clean_scrub_stamp":"2024-09-14T22:56:19.562288+0000","objects_scrubbed":42,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T01:41:42.037199+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0.001128364,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":11616,"num_read_kb":30610,"num_write":481,"num_write_kb":42134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"189'649","reported_seq":5757,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:27.019483+0000","last_change":"2024-09-14T22:56:27.019530+0000","last_active":"2024-09-14T22:56:27.019483+0000","last_peered":"2024-09-14T22:56:27.019483+0000","last_clean":"2024-09-14T22:56:27.019483+0000","last_became_active":"2024-09-14T22:55:51.142779+0000","last_became_peered":"2024-09-14T22:55:51.142779+0000","last_unstale":"2024-09-14T22:56:27.019483+0000","last_undegraded":"2024-09-14T22:56:27.019483+0000","last_fullsized":"2024-09-14T22:56:27.019483+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'649","last_scrub_stamp":"2024-09-14T22:56:27.019373+0000","last_deep_scrub":"189'649","last_deep_scrub_stamp":"2024-09-14T22:56:27.019373+0000","last_clean_scrub_stamp":"2024-09-14T22:56:27.019373+0000","objects_scrubbed":36,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T23:27:50.752669+0000","scrub_duration":27,"objects_trimmed":24,"snaptrim_duration":0.041728379000000003,"stat_sum":{"num_bytes":0,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":36,"num_read":6204,"num_read_kb":34738,"num_write":712,"num_write_kb":24203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"189'457","reported_seq":1683,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:19.077858+0000","last_change":"2024-09-14T22:56:19.077920+0000","last_active":"2024-09-14T22:56:19.077858+0000","last_peered":"2024-09-14T22:56:19.077858+0000","last_clean":"2024-09-14T22:56:19.077858+0000","last_became_active":"2024-09-14T22:55:56.767773+0000","last_became_peered":"2024-09-14T22:55:56.767773+0000","last_unstale":"2024-09-14T22:56:19.077858+0000","last_undegraded":"2024-09-14T22:56:19.077858+0000","last_fullsized":"2024-09-14T22:56:19.077858+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'457","last_scrub_stamp":"2024-09-14T22:56:19.077787+0000","last_deep_scrub":"189'457","last_deep_scrub_stamp":"2024-09-14T22:56:19.077787+0000","last_clean_scrub_stamp":"2024-09-14T22:56:19.077787+0000","objects_scrubbed":47,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T02:01:29.843095+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0.0013471519999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":1274,"num_read_kb":10604,"num_write":428,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"189'1034","reported_seq":6852,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-14T22:56:28.050026+0000","last_change":"2024-09-14T22:56:28.050052+0000","last_active":"2024-09-14T22:56:28.050026+0000","last_peered":"2024-09-14T22:56:28.050026+0000","last_clean":"2024-09-14T22:56:28.050026+0000","last_became_active":"2024-09-14T22:55:51.143935+0000","last_became_peered":"2024-09-14T22:55:51.143935+0000","last_unstale":"2024-09-14T22:56:28.050026+0000","last_undegraded":"2024-09-14T22:56:28.050026+0000","last_fullsized":"2024-09-14T22:56:28.050026+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'1034","last_scrub_stamp":"2024-09-14T22:56:28.049945+0000","last_deep_scrub":"189'1034","last_deep_scrub_stamp":"2024-09-14T22:56:28.049945+0000","last_clean_scrub_stamp":"2024-09-14T22:56:28.049945+0000","objects_scrubbed":51,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T08:11:10.151634+0000","scrub_duration":43,"objects_trimmed":16,"snaptrim_duration":0.034758323000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":8863,"num_read_kb":12560,"num_write":1145,"num_write_kb":21655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1381,"num_object_clones":0,"num_object_copies":2762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1381,"num_whiteouts":1377,"num_read":260757,"num_read_kb":1066701,"num_write":19012,"num_write_kb":702896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":141629825,"num_keys_recovered":114,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},"log_size":20201,"ondisk_log_size":20201,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542194,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56196,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315644,"statfs":{"total":96636764160,"available":96579219456,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542195,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29248,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342592,"statfs":{"total":96636764160,"available":96606814208,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542194,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55620,"kb_used_data":2228,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316220,"statfs":{"total":96636764160,"available":96579809280,"internally_reserved":0,"allocated":2281472,"data_stored":1113541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":169,"seq":725849473035,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35780,"kb_used_data":2228,"kb_used_omap":47,"kb_used_meta":33488,"kb_avail":94336060,"statfs":{"total":96636764160,"available":96600125440,"internally_reserved":0,"allocated":2281472,"data_stored":1113541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49036,"internal_metadata":34291828},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542194,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33408,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338432,"statfs":{"total":96636764160,"available":96602554368,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41728,"kb_used_data":2224,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330112,"statfs":{"total":96636764160,"available":96594034688,"internally_reserved":0,"allocated":2277376,"data_stored":1113522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29256,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342584,"statfs":{"total":96636764160,"available":96606806016,"internally_reserved":0,"allocated":2285568,"data_stored":1113560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60356,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311484,"statfs":{"total":96636764160,"available":96574959616,"internally_reserved":0,"allocated":2871296,"data_stored":1703890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21018,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T22:56:38.711 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-14T22:56:39.006 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-14T22:56:39.006 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-14T22:56:39.006 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T22:56:39.006 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.042 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T22:56:39.042 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-14T22:56:39.042 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.109 INFO:tasks.ceph.osd.1:Stopped 2024-09-14T22:56:39.109 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T22:56:39.109 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.178 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T22:56:39.178 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-14T22:56:39.178 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.256 INFO:tasks.ceph.osd.3:Stopped 2024-09-14T22:56:39.256 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T22:56:39.256 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.287 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T22:56:39.288 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-14T22:56:39.288 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.367 INFO:tasks.ceph.osd.5:Stopped 2024-09-14T22:56:39.367 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-14T22:56:39.367 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.439 INFO:tasks.ceph.osd.6:Stopped 2024-09-14T22:56:39.439 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-14T22:56:39.440 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.514 INFO:tasks.ceph.osd.7:Stopped 2024-09-14T22:56:39.514 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-14T22:56:39.514 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-14T22:56:39.514 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.538 INFO:tasks.ceph.mgr.y:Stopped 2024-09-14T22:56:39.538 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-14T22:56:39.538 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.584 INFO:tasks.ceph.mgr.x:Stopped 2024-09-14T22:56:39.584 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-14T22:56:39.584 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-14T22:56:39.584 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.595 INFO:tasks.ceph.mon.a:Stopped 2024-09-14T22:56:39.595 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-14T22:56:39.595 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.646 INFO:tasks.ceph.mon.c:Stopped 2024-09-14T22:56:39.647 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-14T22:56:39.647 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T22:56:39.659 INFO:tasks.ceph.mon.b:Stopped 2024-09-14T22:56:39.659 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-14T22:56:39.660 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-14T22:56:39.724 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:56:39.725 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-14T22:56:40.012 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:56:40.013 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-14T22:56:40.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:56:40.101 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-14T22:56:40.158 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:56:40.158 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-14T22:56:40.222 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:56:40.222 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-14T22:56:40.595 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:56:40.595 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-14T22:56:40.681 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:56:40.681 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-14T22:56:40.731 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:56:40.731 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-14T22:56:40.785 INFO:tasks.ceph:Archiving mon data... 2024-09-14T22:56:40.786 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/data/mon.a.tgz 2024-09-14T22:56:40.787 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-09-14T22:56:40.802 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.OAn0K7oSjh 2024-09-14T22:56:40.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.OAn0K7oSjh 2024-09-14T22:56:40.960 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.OAn0K7oSjh 2024-09-14T22:56:41.044 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.OAn0K7oSjh is 414KB 2024-09-14T22:56:41.113 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.OAn0K7oSjh 2024-09-14T22:56:41.128 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/data/mon.c.tgz 2024-09-14T22:56:41.128 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-09-14T22:56:41.182 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.jMyQYBUrUI 2024-09-14T22:56:41.182 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.jMyQYBUrUI 2024-09-14T22:56:41.364 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.jMyQYBUrUI 2024-09-14T22:56:41.446 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.jMyQYBUrUI is 564KB 2024-09-14T22:56:41.544 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.jMyQYBUrUI 2024-09-14T22:56:41.559 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/data/mon.b.tgz 2024-09-14T22:56:41.559 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-14T22:56:41.577 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.eucWYKH4pG 2024-09-14T22:56:41.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.eucWYKH4pG 2024-09-14T22:56:41.776 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.eucWYKH4pG 2024-09-14T22:56:41.860 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.eucWYKH4pG is 565KB 2024-09-14T22:56:41.957 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.eucWYKH4pG 2024-09-14T22:56:41.972 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-14T22:56:41.972 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T22:56:41.975 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T22:56:42.054 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T22:56:42.092 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T22:56:42.092 INFO:tasks.ceph:Archiving crash dumps... 2024-09-14T22:56:42.093 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/remote/smithi076/crash 2024-09-14T22:56:42.094 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T22:56:42.119 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/remote/smithi114/crash 2024-09-14T22:56:42.120 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T22:56:42.156 INFO:tasks.ceph:Compressing logs... 2024-09-14T22:56:42.156 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T22:56:42.164 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T22:56:42.253 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33420.log 2024-09-14T22:56:42.253 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33636.log 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33420.log.gz 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.48568.log 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33636.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33636.log.gz 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/objectstore_tool.48568.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33556.log 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.45928.log 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33556.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.46826.log 2024-09-14T22:56:42.254 INFO:teuthology.orchestra.run.smithi114.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33556.log.gz 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/objectstore_tool.45928.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/objectstore_tool.46826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33460.log 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33460.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-osd.5.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33460.log.gz 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph.log: 85.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mgr.x.log: 94.9% -- replaced with /var/log/ceph/objectstore_tool.48568.log.gz 2024-09-14T22:56:42.255 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.47668.log 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-mon.b.log: 94.9% -- replaced with /var/log/ceph/objectstore_tool.45928.log.gz 95.3% -- replaced with /var/log/ceph/objectstore_tool.46826.log.gz 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33676.log 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/objectstore_tool.47668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33516.log 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33676.log.gz 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33596.log 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33516.log.gz 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33716.log 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33596.log.gz 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.33716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33716.log.gz 2024-09-14T22:56:42.256 INFO:teuthology.orchestra.run.smithi114.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.47668.log.gz 2024-09-14T22:56:42.314 INFO:teuthology.orchestra.run.smithi114.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-14T22:56:42.336 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-14T22:56:42.336 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51845.log 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60904.log 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38807.log 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51845.log.gz 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54052.log 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60904.log.gz 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62046.log 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38807.log.gz 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54052.log.gz 2024-09-14T22:56:42.337 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.62046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59522.log 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62046.log.gz 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60967.log 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56227.log 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59522.log.gz 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38799.log 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60967.log.gz 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50024.log 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56227.log.gz 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52465.log 2024-09-14T22:56:42.338 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38799.log.gz 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59309.log 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50024.log.gz 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61542.log 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52465.log.gz 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38804.log 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59309.log.gz 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61542.log.gz 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61500.log 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59946.log 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38804.log.gz 2024-09-14T22:56:42.339 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62130.log 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61500.log.gz 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46325.log 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59946.log.gz 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.62130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52590.log 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62130.log.gz 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58275.log 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46325.log.gz 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44562.log 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52590.log.gz 2024-09-14T22:56:42.340 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.58275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33453.log 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58275.log.gz 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56935.log 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44562.log.gz 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53825.log 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33453.log.gz 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50415.log 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56935.log.gz 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53825.log.gz 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54833.log 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45506.log 2024-09-14T22:56:42.341 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50415.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50415.log.gz 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45102.log 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54833.log.gz 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39717.log 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45506.log.gz 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56860.log 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45102.log.gz 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39717.log.gz 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39471.log 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41476.log 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56860.log.gz 2024-09-14T22:56:42.342 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39471.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.61962.log 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39471.log.gz 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55958.log 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41476.log.gz 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61962.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 0.0% 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61962.log.gz 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60627.log 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55958.log.gz 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39885.log 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41076.log 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60627.log.gz 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49268.log 2024-09-14T22:56:42.343 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39885.log.gz 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59647.log 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41076.log.gz 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49268.log.gz 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57875.log 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48061.log 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59647.log.gz 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57875.log.gzgzip 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42310.log 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39478.log 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48061.log.gz 2024-09-14T22:56:42.344 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42310.log.gz 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51563.log 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40552.log 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39478.log.gz 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60905.log 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57795.log 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40552.log.gz 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60905.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.60905.log.gz 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60623.log 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47431.log 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57795.log.gz 2024-09-14T22:56:42.345 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53245.log 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60623.log.gz 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55817.log 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47431.log.gz 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53245.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53245.log.gz -5 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60030.log 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45970.log 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55817.log.gz 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60030.log.gz 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59988.log 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60906.log 2024-09-14T22:56:42.346 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45970.log.gz 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59988.log.gz 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56773.log 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39129.log 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60906.log.gz 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56773.log.gz 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44159.log 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49518.log 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39129.log.gz 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44159.log.gz 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39142.log 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41834.log 2024-09-14T22:56:42.347 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49518.log.gz 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39142.log.gz 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39138.log 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51310.log 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41834.log.gz 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39138.log.gz 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62172.log 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56147.log 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51310.log.gz 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.62172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39989.log 2024-09-14T22:56:42.348 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62172.log.gz 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56147.log.gz 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57835.log 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39989.log.gz 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57835.log.gz 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57342.log 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42189.log 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57342.log.gz 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38583.log 2024-09-14T22:56:42.349 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38803.log 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42189.log.gz 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38583.log.gz 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48364.log 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55375.log 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38803.log.gz 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48364.log.gz 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55375.log.gz 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40796.log 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62088.log 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2024-09-14T22:56:42.350 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40796.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.40796.log.gz 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61710.log 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.62088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61794.log 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62088.log.gz 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61710.log.gz 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39339.log 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59820.log 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61794.log.gz 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38541.log 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39339.log.gz 2024-09-14T22:56:42.351 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59439.log 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59820.log.gz 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61374.log 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38541.log.gz 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47700.log 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59439.log.gz 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55014.log 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61374.log.gz 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47700.log.gz 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55014.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.55014.log.gz 2024-09-14T22:56:42.352 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50110.log 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39115.log 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45014.log 0.0% 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50110.log.gz 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54596.log 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39115.log.gz 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45014.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45014.log.gz 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40973.log 2024-09-14T22:56:42.353 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54596.log.gz 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50174.log 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40973.log.gz 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46835.log 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62212.log 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50174.log.gz 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46835.log.gz 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49125.log 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57915.log 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.62212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62212.log.gz 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49125.log.gz 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55694.log 2024-09-14T22:56:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50641.log 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57915.log.gz 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.55694.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39143.log 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55694.log.gz 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50641.log.gz 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60240.log 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43336.log 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39143.log.gz 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60240.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60240.log.gz -5 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40614.log 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60450.log 2024-09-14T22:56:42.355 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43336.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43336.log.gz 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40614.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40614.log.gz -5 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61248.log 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58488.log 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60450.log.gz 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58215.log 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61248.log.gz 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.58488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41963.log 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58488.log.gz 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.58215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61458.log 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58215.log.gz 2024-09-14T22:56:42.356 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61836.log 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41963.log.gz 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57590.log 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61458.log.gz 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47270.log 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61836.log.gz 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61206.log 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57590.log.gz 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49045.log 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47270.log.gz 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48854.log 2024-09-14T22:56:42.357 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61206.log.gz 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50688.log 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49045.log.gz 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44621.log 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48854.log.gz 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61878.log 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50688.log.gz 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33349.log 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44621.log.gz 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61878.log.gz 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51718.log 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55877.log 2024-09-14T22:56:42.358 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33349.log.gz 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60324.log 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51718.log.gz 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48633.log 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55877.log.gz 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39141.log 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60324.log.gz 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38796.log 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48633.log.gz 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47078.log 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39141.log.gz 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38490.log 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38796.log.gz 2024-09-14T22:56:42.359 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59689.log 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47078.log.gz 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60908.log 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38490.log.gz 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59689.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.59689.log.gz 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.57118.log 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60908.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60908.log.gz 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48104.log 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57118.log.gz 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60629.log 2024-09-14T22:56:42.360 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61668.log 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48104.log.gz 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49430.log 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60629.log.gz 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49981.log 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61668.log.gz 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49430.log.gz 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60619.log 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61584.log 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49981.log.gz 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60619.log.gz 2024-09-14T22:56:42.361 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43418.log 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.61584.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52548.log 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61584.log.gz 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61626.log 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43418.log.gz 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52548.log.gzgzip 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39474.log 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44671.log 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61626.log.gz 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39474.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39474.log.gz 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43278.log 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41036.log 2024-09-14T22:56:42.362 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44671.log.gz 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43278.log.gz 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55797.log 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39476.log 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41036.log.gz 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55797.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.46789.log -- replaced with /var/log/ceph/ceph-client.admin.55797.log.gz 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48980.log 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39476.log.gz 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52388.log 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46789.log.gz 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53531.log 2024-09-14T22:56:42.363 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48980.log.gz 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52388.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.52388.log.gz 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60966.log 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53531.log.gz 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38806.log 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60966.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.60966.log.gz --verbose 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.45445.log 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42739.log 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38806.log.gz 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45445.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45445.log.gz 2024-09-14T22:56:42.364 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40389.log 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56087.log 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42739.log.gz 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40389.log.gz 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56087.log.gz 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48591.log 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61416.log 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48591.log.gz 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48281.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz -5 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55422.log 2024-09-14T22:56:42.365 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61920.log 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61416.log.gz 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55422.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55422.log.gz 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40969.log 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61920.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61920.log.gz 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40969.log.gz 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40127.log 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.50104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52485.log 2024-09-14T22:56:42.366 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55918.log 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40127.log.gz 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52485.log.gz 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46917.log 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55918.log.gz 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52656.log 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gz 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54241.log 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46917.log.gz 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61164.log 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52656.log.gz 2024-09-14T22:56:42.367 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57734.log 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54241.log.gz 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49559.log 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61164.log.gz 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57734.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57734.log.gz 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.49559.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46218.log 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49559.log.gz 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46218.log.gz 2024-09-14T22:56:42.368 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58585.log 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60632.log 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.58585.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58585.log.gz 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45299.log 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60632.log.gz 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39477.log 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45299.log.gz 2024-09-14T22:56:42.369 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33056.log 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39477.log.gz 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39472.log 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.33056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33056.log.gz 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60198.log 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39472.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39472.log.gz 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.41898.log 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60198.log.gz 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61752.log 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41898.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53884.log 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41898.log.gz 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2024-09-14T22:56:42.370 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61752.log.gz 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.53884.log: -- /var/log/ceph/ceph-client.admin.45911.log 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53884.log.gz 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54657.log: -- /var/log/ceph/ceph-client.admin.51007.log 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45911.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45911.log.gz 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61290.log 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51007.log.gz 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48453.log 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61290.log: gzip 28.2% -- replaced with /var/log/ceph/ceph-client.admin.61290.log.gz 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59128.log 2024-09-14T22:56:42.371 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48453.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48453.log.gz 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42452.log 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59128.log.gz 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53142.log 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42452.log.gz 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53142.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43900.log 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53142.log.gz 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45593.log 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41149.log 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45593.log: /var/log/ceph/ceph-client.admin.43900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43900.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45593.log.gz 2024-09-14T22:56:42.372 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58954.log 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41149.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60072.log 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41149.log.gz 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62004.log 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.58954.log: /var/log/ceph/ceph-client.admin.60072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58954.log.gz 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60072.log.gz 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56454.log 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.62004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62004.log.gz 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58775.log 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56454.log.gz 2024-09-14T22:56:42.373 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41432.log 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.58775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58775.log.gz 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39927.log 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41432.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47806.log 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.39927.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39927.log.gz 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr: /var/log/ceph/ceph-client.admin.52942.log 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47806.log.gz 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58609.log 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52942.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52942.log.gz 2024-09-14T22:56:42.374 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56528.log 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.58609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58609.log.gz 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39675.log 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56528.log.gz 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50854.log 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39675.log.gz 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48180.log 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50854.log.gz 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59313.log 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48180.log.gz 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51655.log 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59313.log.gz 2024-09-14T22:56:42.375 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43778.log 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51655.log.gz 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56047.log 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43778.log.gz 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44001.log 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56047.log.gz 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60895.log 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44001.log.gz 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52082.log 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60895.log.gz 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38465.log 2024-09-14T22:56:42.376 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52082.log.gz 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60366.log 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38465.log.gz 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43478.log 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60366.log.gz 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60968.log 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43478.log.gz 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49637.log 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60968.log.gz 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44753.log 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49637.log.gz 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60408.log 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44753.log.gz 2024-09-14T22:56:42.377 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44516.log 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60408.log.gz 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60628.log 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44516.log.gz 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60628.log.gz 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42451.log 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42451.log.gz 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50234.log 2024-09-14T22:56:42.378 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51613.log 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50234.log.gz 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45676.log 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51613.log.gz 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40042.log 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45676.log.gz 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46437.log 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.40042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39759.log 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46437.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44894.log 2024-09-14T22:56:42.379 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.0.40042.log.gz 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38409.log 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39759.log: /var/log/ceph/ceph-client.admin.44894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39759.log.gz 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39479.log 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44894.log.gz 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42996.log 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38409.log.gz 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59778.log 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39479.log: /var/log/ceph/ceph-client.admin.42996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60625.log 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42996.log.gz 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39479.log.gz 2024-09-14T22:56:42.380 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59778.log.gz 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42880.log 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60625.log.gz 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61332.log 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42880.log.gz 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27220.log 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.61332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61332.log.gz 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57235.log 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.tmp-client.admin.27220.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27220.log.gz 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52057.log 2024-09-14T22:56:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57235.log.gz 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55090.log 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52057.log.gz 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57566.log 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.55090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55090.log.gz 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59862.log 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57566.log.gz 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47200.log 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59862.log.gz 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60114.log 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47200.log.gz 2024-09-14T22:56:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53462.log 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60114.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60114.log.gz 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60282.log 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53462.log.gz 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50517.log 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60282.log.gz 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50517.log.gz 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59904.log 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53412.log 2024-09-14T22:56:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59904.log.gz 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.53412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53412.log.gz 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40307.log 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57674.log 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40307.log.gz 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60618.log 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57674.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57674.log.gz 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42821.log 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60618.log.gz 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59736.log 2024-09-14T22:56:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42821.log.gz 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41671.log 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59736.log.gz 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41671.log.gz 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40129.log 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40125.log 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40129.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39843.log 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40129.log.gz 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40125.log.gz 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39137.log 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38247.log 2024-09-14T22:56:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39843.log.gz 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39137.log.gz 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39801.log 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43197.log 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39801.log: /var/log/ceph/ceph-client.admin.38247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39801.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38247.log.gz 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38802.log 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54284.log 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43197.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43197.log.gz 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38802.log.gz 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57964.log 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38625.log 2024-09-14T22:56:42.386 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.54284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54284.log.gz 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.57964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57964.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56003.log 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43721.log 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38625.log.gz 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.56003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59605.log 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56003.log.gz 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60156.log 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.59605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43721.log.gz 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59605.log.gz 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41964.log 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.60156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60156.log.gz/var/log/ceph/ceph-client.admin.41964.log: 2024-09-14T22:56:42.387 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41964.log.gz 2024-09-14T22:56:42.388 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-14T22:56:42.403 INFO:teuthology.orchestra.run.smithi114.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-14T22:56:42.600 INFO:teuthology.orchestra.run.smithi076.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-14T22:56:43.009 INFO:teuthology.orchestra.run.smithi076.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-14T22:56:43.887 INFO:teuthology.orchestra.run.smithi114.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-14T22:56:44.615 INFO:teuthology.orchestra.run.smithi076.stderr: 96.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-14T22:56:45.537 INFO:teuthology.orchestra.run.smithi076.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-14T22:56:45.607 INFO:teuthology.orchestra.run.smithi076.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-14T22:56:45.609 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-14T22:56:45.609 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m3.433s 2024-09-14T22:56:45.609 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m10.034s 2024-09-14T22:56:45.609 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.548s 2024-09-14T22:56:45.656 INFO:teuthology.orchestra.run.smithi114.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-14T22:56:46.471 INFO:teuthology.orchestra.run.smithi114.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-14T22:56:46.948 INFO:teuthology.orchestra.run.smithi114.stderr: 96.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-14T22:56:46.950 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-14T22:56:46.950 INFO:teuthology.orchestra.run.smithi114.stderr:real 0m4.732s 2024-09-14T22:56:46.950 INFO:teuthology.orchestra.run.smithi114.stderr:user 0m13.707s 2024-09-14T22:56:46.950 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m0.615s 2024-09-14T22:56:46.950 INFO:tasks.ceph:Archiving logs... 2024-09-14T22:56:46.951 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/remote/smithi076/log 2024-09-14T22:56:46.952 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T22:56:48.312 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/remote/smithi114/log 2024-09-14T22:56:48.313 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T22:56:49.944 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-14T22:56:49.954 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-14T22:56:49.954 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T22:56:49.957 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T22:56:50.047 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-14T22:56:50.057 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-14T22:56:50.057 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T22:56:50.060 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T22:56:50.074 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-09-14T22:56:50.077 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-14T22:56:50.079 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T22:56:50.079 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-09-14T22:56:50.079 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +810us[ +819us] +/- 49ms 2024-09-14T22:56:50.079 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 +806us[ +815us] +/- 64ms 2024-09-14T22:56:50.080 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -2214us[-2205us] +/- 72ms 2024-09-14T22:56:50.080 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:56:50.082 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T22:56:50.082 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-14T22:56:50.082 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 +839us[ +849us] +/- 49ms 2024-09-14T22:56:50.082 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 +820us[ +830us] +/- 64ms 2024-09-14T22:56:50.082 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -2188us[-2188us] +/- 72ms 2024-09-14T22:56:50.082 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T22:56:50.083 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-14T22:56:50.108 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-14T22:56:50.109 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-14T22:56:50.137 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-14T22:56:50.222 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-14T22:56:50.300 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-14T22:56:50.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:56:50.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-14T22:56:50.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:56:50.363 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-14T22:56:50.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-14T22:56:50.395 INFO:teuthology.task.internal:Duration was 648.941844 seconds 2024-09-14T22:56:50.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-14T22:56:50.437 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-14T22:56:50.437 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T22:56:50.440 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T22:56:50.477 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T22:56:50.482 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T22:56:50.687 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-14T22:56:50.687 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-09-14T22:56:50.687 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T22:56:50.721 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-09-14T22:56:50.721 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T22:56:50.752 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-14T22:56:50.752 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T22:56:50.764 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T22:56:50.820 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-14T22:56:50.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T22:56:50.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T22:56:50.976 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-14T22:56:50.986 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-14T22:56:50.986 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T22:56:51.011 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T22:56:51.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-14T22:56:51.055 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T22:56:51.057 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T22:56:51.078 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-09-14T22:56:51.099 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-09-14T22:56:51.147 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T22:56:51.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:56:51.162 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T22:56:51.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T22:56:51.202 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-14T22:56:51.212 INFO:teuthology.task.internal:Transferring archived files... 2024-09-14T22:56:51.212 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/remote/smithi076 2024-09-14T22:56:51.213 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T22:56:51.252 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-14_21:16:02-rbd-squid-distro-default-smithi/7904734/remote/smithi114 2024-09-14T22:56:51.253 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T22:56:51.286 INFO:teuthology.task.internal:Removing archive directory... 2024-09-14T22:56:51.286 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T22:56:51.288 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T22:56:51.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-14T22:56:51.346 INFO:teuthology.task.internal:Not uploading archives. 2024-09-14T22:56:51.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-14T22:56:51.435 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-14T22:56:51.436 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T22:56:51.438 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T22:56:51.450 INFO:teuthology.orchestra.run.smithi076.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 22:56 /home/ubuntu/cephtest 2024-09-14T22:56:51.453 INFO:teuthology.orchestra.run.smithi114.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 22:56 /home/ubuntu/cephtest 2024-09-14T22:56:51.454 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-14T22:56:51.495 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-14T22:56:51.590 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-hybrid supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} duration: 648.9418435096741 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-14T22:56:51.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T22:56:51.687 INFO:teuthology.run:pass